There are many tools for tidying data in Pandas and the subsequent child nodes will cover the specific usage of the following concepts:
- Data Selection
- Data Merging
- Data Grouping
- Missing Data 
- Date Time Conversions

Tools for Tidying Data

Pandas is a well designed library for raw data processing and manipulation and works very well with other packages such as matplotlib and numpy. Pandas has built in read functions that work for a large variety of data file types, ranging from CSV to html to SPSS. Data that is read in will be converted to a data frame object, which can easily be converted then to other formats , i.e. from a SQL query to a CSV file. 

Usage: ```read_csv(path/filename)```

this will return a data frame object which can be stored in a variable for later use. Other file types will just require the _csv to be changed to _html etc. If the file is included in the same directory as the python program, just the filename as a string passed in as the first parameter will be sufficient. For more details visit:

https://pandas.pydata.org/docs/user_guide/io.html

Data Processing in Pandas

Pandas is built to work very well with the date time library, which can be imported via ```import datetime ```. The date time library provides tools to convert strings into date time objects which can then be used to compare dates and sort objects based on time. This is an important conversion to make in order to perform analysis like time series. 

Not only can pandas convert between strings to date-time objects, it can also provide date time conversions for exisiting date time objects based on time zones. i.e if we are given a dataset set in time zone UTC +5, we can convert it to UTC +8 within as needed.

Date Time

Data visualization will mostly be handled by other libraries, i.e. matplotlib which provide a more complete suite of functions to handle data. However, it is important that data is cleaned properly before being passed to visualization methods, i.e. column names are correct and the data is selected correctly otherwise it may lead to unexpected behavior especially when the data sets grow very large. 

Data Visualization

Pandas provides a large suite of functions designed for exploratory analysis and data manipulation. These include:
- Pivot Tables
- Multi-Indexing
- Merging / Concatenation
- Date-Time
- Visualization
- Sub-Sets

University of Michigan - Ann Arbor

Pandas is a Python library for data manipulation and analysis.
Example import: `import pandas as pd`

Pandas library in Python

Pandas Documentation

The key data structure pandas supports, which is basically like a spreadsheet table with rows and named columns.

It is also called a 2 dimensional data structure

Pandas Dataframe

What does the following code do?

The Pandas Library allows the user to analyze big data and make an educated conclusion on statistical data. (i.e data sets)

Pandas - What is it Useful?

If Python and PIP are already installed on your computer then simply do the following in the terminal:

- pip install pandas

If not, then simply check out the Anaconda Documentation for how to set up .

https://docs.continuum.io/anaconda/install/



Installation of Pandas

What will the code output?

Pandas uses the plot() to create diagrams but Python Pyplot to actually display the digram.

To get started :
- import pandas as pd
- import matplotlib.pyplot as plt
- import seaborn as sns

Learn Before

Related

Learn After