Pandas From Dict

  1. Note that the Index is first build with the keys from the dictionary. After this the Series is reindexed with the given Index values, hence we get all NaN as a result.
  2. I hope that the above tutorial on how to create a DataFrame or Series from a list or a dictionary was useful to you. Pandas and DataFrames are such powerful, flexible tools, and I personally find these to be good methods to create a DataFrame or Series to manipulate. Once you start to generate your own DataFrames, you will also start to see when each of thes.

Pandas.DataFrame.fromdict¶ classmethod DataFrame. Fromdict (data, orient = 'columns', dtype = None, columns = None) source ¶. Construct DataFrame from dict of array-like or dicts. Pandas.DataFrame(data=None, index=None, columns=None, dtype=None, copy=False) But if we are passing a dictionary in data, then it should contain a list like objects in value field like Series, arrays or lists etc i.e. # Dictionary with list object in values.

Creating a new column to a dataframe is a common task in doing data analysis. And this task often comes in a variety of forms. Earlier we saw how to add a column using an existing columns in two ways. In this post we will learn how to add a new column using a dictionary in Pandas.

Pandas library in Python has a really cool function called map that lets you manipulate your pandas data frame much easily. Pandas’ map function lets you add a new column with values from a dictionary if the data frame has a column matching the keys in the dictionary.

Python dict from pandas

Adding a New Column Using keys from Dictionary matching a column in pandas

Let us say you have pandas data frame created from two lists as columns; continent and mean_lifeExp.

and let us say we also have a dictionary, where the keys are “continent” as in the above data frame and values are mean population over years.

Note that it is a dictionary, so the order of items do not match the continent column of the data frame.

Map function to Add a New Column to pandas with Dictionary

Let us say we want to add a new column ‘pop’ in the pandas data frame with values from the dictionary. Note the keys of the dictionary are “continents” and the column “continent” in the data frame. Pandas’ map function is here to add a new column in pandas dataframe using the keys:values from the dictionary.

Pandas From Dict Index

Voila!! here is the updated data frame with a new column from the dictionary.

Another common use of dictionary to add a new column in Pandas is to code an exisiting column using dictionary and create a new column. For example, let us consider the gapminder data frame


Related posts:

Here is yet another example of how useful and powerful Pandas is. Pandas can create dataframes from many kinds of data structures—without you having to write lots of lengthy code. One of those data structures is a dictionary.

In this tutorial, we show you two approaches to doing that.

Pandas Dataframe From Dict

(This tutorial is part of our Pandas Guide. Use the right-hand menu to navigate.)

A word on Pandas versions

Before you start, upgrade Python to at least 3.7. With Python 3.4, the highest version of Pandas available is 0.22, which does not support specifying column names when creating a dictionary in all cases.

If you are running virtualenv, create a new Python environment and install Pandas like this:

You can check the Pandas version with:

Create dataframe with Pandas DataFrame constructor

Here we construct a Pandas dataframe from a dictionary. We use the Pandas constructor, since it can handle different types of data structures.

The dictionary below has two keys, scene and facade. Each value has an array of four elements, so it naturally fits into what you can think of as a table with 2 columns and 4 rows.

Pandas is designed to work with row and column data. Each row has a row index. By default, it is the numbers 0, 1, 2, 3, … But it also lets you use names.

So, let’s use the same in the array idx.


Here is the resulting dataframe:

Create dataframe with Pandas from_dict() Method

Pandas also has a Pandas.DataFrame.from_dict() method. If that sounds repetitious, since the regular constructor works with dictionaries, you can see from the example below that the from_dict() method supports parameters unique to dictionaries.

In the code, the keys of the dictionary are columns. The row indexes are numbers. That is default orientation, which is orient=’columns’ meaning take the dictionary keys as columns and put the values in rows.

Now we flip that on its side. We will make the rows the dictionary keys.

Notice that the columns have no names, only numbers. That’s not very useful, so below we use the columns parameter, which was introduced in Pandas 0.23.

It’s as simple as putting the column names in an array and passing it as the columns parameter. One wonders why the earlier versions of Pandas did not have that.

Related reading

  • Apache Spark Guide, a series of tutorials