Pandas is a well designed library for raw data processing and manipulation and works very well with other packages such as matplotlib and numpy. Pandas has built in read functions that work for a large variety of data file types, ranging from CSV to html to SPSS. Data that is read in will be converted to a data frame object, which can easily be converted then to other formats , i.e. from a SQL query to a CSV file. 

Usage: ```read_csv(path/filename)```

this will return a data frame object which can be stored in a variable for later use. Other file types will just require the _csv to be changed to _html etc. If the file is included in the same directory as the python program, just the filename as a string passed in as the first parameter will be sufficient. For more details visit:

https://pandas.pydata.org/docs/user_guide/io.html

University of Michigan - Ann Arbor

Pandas provides a large suite of functions designed for exploratory analysis and data manipulation. These include:
- Pivot Tables
- Multi-Indexing
- Merging / Concatenation
- Date-Time
- Visualization
- Sub-Sets

Exploratory Data Analysis in Pandas

There are many tools for tidying data in Pandas and the subsequent child nodes will cover the specific usage of the following concepts:
- Data Selection
- Data Merging
- Data Grouping
- Missing Data 
- Date Time Conversions

Tools for Tidying Data

Data Processing in Pandas

Pandas is built to work very well with the date time library, which can be imported via ```import datetime ```. The date time library provides tools to convert strings into date time objects which can then be used to compare dates and sort objects based on time. This is an important conversion to make in order to perform analysis like time series. 

Not only can pandas convert between strings to date-time objects, it can also provide date time conversions for exisiting date time objects based on time zones. i.e if we are given a dataset set in time zone UTC +5, we can convert it to UTC +8 within as needed.

Date Time

Data visualization will mostly be handled by other libraries, i.e. matplotlib which provide a more complete suite of functions to handle data. However, it is important that data is cleaned properly before being passed to visualization methods, i.e. column names are correct and the data is selected correctly otherwise it may lead to unexpected behavior especially when the data sets grow very large. 

Learn Before

Related