How to Read Csv File With Different Number of Columns Python
Introduction to CSV files
CSV stands for comma separated values. It is one of most common files that is used to transfer and store data. Information technology mostly stores tabular data ( numbers and text) in plain text. Each line in CSV files records information. And each record consists of one or more than one field, separated by commas. The ability to read, manipulate, and write information to and from CSV files using Python is a key skill to master for any information scientist or concern annotator. In this tutorial, we will comprehend everything that you demand to master with CSV files using Python. Nosotros will see how Python helps us to create, write and read CSV files. We will also cover some basic libraries of Python that are very powerful for working with CSV files.
Python open() CSV file
Python has a born office open() to open files. This function returns a file object which is then used to read or modify accordingly. We tin apply this role to likewise open up a CSV file type.
See the example below:
>> f = open("myfile.csv") >> f = open(myfile.text") If the file is not in the current directory, the we have to provide full path to open up it.
>> f = open ("C:\Users\Documents\CSVfile\myfile.csv") Different modes to open CSV file in Python
We can also specify the fashion of opening the file. Specifying mode means, specifying whether we want to read "r" , write "due west" or append "a" to the file. Python also provides a mode to specify whether we want to open a file in binary mode or text mode.
If we will not provide whatever way explicitly, then by default, it will exist read in text manner. On the other hand binary mode returns bytes and this way is used to bargain with non-texted files.
The following table specifies the symbols used for dissimilar manner:
-
"r"opens file in reading mode (default as well). -
"w"opens file for writing and create new ane if it does not exist. -
"x"opens file for exclusive cosmos. If the file already exists, the functioning fails. -
"t"opens file in text mode( default also). -
"b"opens file in binary way. -
"a"opens file for appending information at the finish of file without removing existing data. If file does not be, it creates a new one. -
"+"use to update file ( writing and reading)
See the examples below:
f = open("myfile.csv") ## This is the aforementioned as "rt" way. f = open("myfile.csv", "westward") ## write in text mode f = open("myimage.png", "r+b") ## read and write in binary manner Python read CSV
reader Object in Python is used to read CSV files. Firstly, Python congenital-in part open() , is used to open a CSV file as a text file, and and then pass it to the reader, which then reads the file.
The case below will help usa to understand reading CSV in more than details. Let say we accept csv file named myfile.csv which contains the post-obit data:
No, Name, Gender ane, Erlan, Male two, Alex, Male person 3, soro, Female 4, Khan, Male Now let'due south come across python code to open and read this csv file.
import csv ## opens csv file with open('myfile.csv','r')as f: data = csv.reader(f) ## Read the csv file and store content of this file into data variable ## Utilize for loop to iterate over the content in data variable for row in data: print(row) In this python example, we used open() role to open a csv file. Then csv.reader is used to read the file and returns an iterable reader object. Python for loop is so used to iterate over the object and print out content in each row.
Output:
Notice that the contents in a row are printed in the form of a list.
Python Write CSV Examples
Writing a csv file in Python is pretty easy. Firstly, we have to open the file in writing "due west" manner by using open() function. So we have to create a CSV writer object by calling python congenital-in function writer(). Now nosotros can write data into our CSV file past calling writerow() or writerows() functions.
Case-1: Overwrite CSV File Content
In this example we will overwrite the content of our myfile.csv. This example tin can likewise be used to write a new CSV file just an empty CSV file should be present for writing. Let's add the post-obit data into our myfile.csv
5,dilover,Male Here is a sample python lawmaking to overwrite the content into our CSV file. We are using write mode to write into the CSV file:
import csv ## define new listing of data newStudent = ["5", "dilover", "Male"] ## open csv file in writing mode with open("myfile.csv", "w") as f: ## stores the data in variable data data = csv.author(f) ## add together data to csv file data.writerow(newStudent) This will add the following data in our csv file only at the same time will remove all the previous data. When nosotros prints the csv file, we gets.
~]# cat myfile.csv 5,dilover,Male Example-2: Append content into CSV file
Since we used write mode in the previous case, the existing information of the CSV file was overwritten. To avoid this and retain the previous data, we have to open up the file in suspend mode:
See the example below:
newStudent = ["5", "dilover", "Male"] ## open csv file in append way with open up("myfile.csv", "a") as f: ## store data in variable data = csv.author(f) ## add data to csv file information.writerow(newStudent) Python close() CSV file
It is ever a skilful practice to shut a CSV file after opening information technology. Python uses close() function to shut CSV file as shown below.
newStudent = ["5", "dilover", "Male"] ## opening csv file in suspend fashion with open("myfile.csv", "a") every bit f: ## stores data in variable data = csv.writer(f) ## add data to csv file data.writerow(newStudent) ## close the opened csv file f.close() Pandas and CSV files
Pandas is a powerful and flexible Python package, that aid us to work with series of data. It as well help us to show our information graphically, contains many powerful statistic methods and many more. One of the of import features of pandas is its power to write and read excel and CSV files. In the following series, we will see how nosotros tin employ pandas to open, write and read CSV files using pandas.
Instance-i: Reading CSV file using pandas
First we have to import the pandas library. If it is not installed, yous can install it using the pip control. The post-obit example shows how to read a CSV file using pandas.
import pandas as pd ## read myfile.csv which is in the same directory csvFile = pd.read_csv('myfile.csv') Pandas will search the file myfile.csv. read_csv() function takes 1 argument which is the proper noun or the full path of the file and other optional arguments as well. We can employ the head() function to see if everything is imported correctly or non.
import pandas equally pd ## csv file is in the aforementioned directory csvFile = pd.read_csv('myfile.csv') ## print the data in csv file impress(csvFile.head()) HINT:
If yous get following error ModuleNotFoundError: No module named 'pandas', it ways pandas module is not available on your server. Apply following control to install pandas module:
~]$ python3 -m pip install pandas
Replace python3 with the your default python binary
Output:
Example-2: How to read specific row from CSV file in Python
Pandas provides us with a more powerful feature by letting u.s. to select and import n number of rows instead of importing the whole CSV file. This feature is helpful when we demand but a limited number of data from a huge file.
read.csv() function takes an optional argument to print n number of rows. See the instance below which prints but the first two rows of the myfile.csv.
import pandas as pd ## selecting beginning 2 rows csvFile = pd.read_csv('myfile.csv', nrows=ii) ## press the selected rows print(csvFile.head()) Output:
~]# python3 eg-3.py No Name Gender 0 one Erlan Male 1 2 Alex Male Example-3: How to delete selected rows of CSV file in Python
Not but from the top, but pandas also helps usa to print the rows from the middle of the data besides. We can as well define the range of rows in read.csv() to import simply rows from a limited range.
See the example which imports only the second and and forth row from myfile.csv and eliminates heading and third row.
import pandas as pd ## skiping rows csvFile = pd.read_csv('myfile.csv',skiprows=(0, 2), nrows=ii) ## printing the data print(csvFile.head()) Output:
~]# python3 eg-3.py 1 Erlan Male 0 three soro Female person i 4 Khan Male Example-iv: How to change header of CSV file in Python
We can alter the header proper noun in a CSV file in Python using the read_csv() part. We can provide the list of names every bit arguments which will deed as the new Header. See the example beneath to understand custom naming of header in CSV file.
import pandas as pd ## giving custom names to columns csvFile = pd.read_csv('myfile.csv',skiprows=(0, one), names=["Number", "FirstName", "G"]) print(csvFile.head()) Output:
Number FirstName K 0 two Alex Male ane 3 soro Female ii four Khan Male Example-5: How to avoid Python/Pandas creating an index in a CSV File
You lot may have noticed that a new column with the index number is generated by pandas. This is the default behaviour while readin CSV file in Python. We tin remove this auto indexing past making Alphabetize to False.
Run into the below instance which disables the auto indexing.
import pandas as pd <i>## giving custom names to columns</i> csvFile = pd.read_csv('myfile.csv',skiprows=(0, 1), names=["Number", "FirstName", "Yard"]) <i># csvFile.reset_index(drib=False)</i> print(csvFile.to_string(index=Faux)) Output:
Example-6: How to read CSV file from URL in Python
Some other powerful feature that pandas provides is reading csv files from URL. We can employ the read_csv() office and laissez passer the URL instead of the path of the file. The following instance demonstrates reading a CSV file from URL.
import pandas every bit pd ## importing data from given url df = pd.read_csv("url of csv file goes here") df.head() Example-vii: How to read specific column from CSV file in Python
We can import specific columns from a csv file past using df[colun_name]. Offset we have to specify a listing consisting of names of the columns and then we can apply df[column_name] to go access to a specific column.
Meet the example below, which shows the elementary syntax to read only the third column of the CSV file:
import pandas as pd ## reading csv file from same directory csvFile = pd.read_csv('myfile.csv', names=["Number", "FirstName", "Gender"]) ## this prints the specific column of CSV file print(csvFile["Gender"]) Output:
~]# python3 eg-half-dozen.py 0 Gender 1 Male 2 Male person three Female four Male Proper noun: Gender, dtype: object This example will only print the 2d column of the CSV file:
import pandas as pd ## reading csv file from same directory csvFile = pd.read_csv('myfile.csv', names=["Number", "FirstName", "Gender"]) ## this prints the specific cavalcade print(csvFile["FirstName"]) Output:
~]# python3 eg-6.py 0 Name 1 Erlan 2 Alex 3 soro four Khan Name: FirstName, dtype: object Example-8: How to access specific rows of CSV file using .loc()
In pandas we can get data from a specific row using its function .loc(). We pass Index position of rows in an integer or list of integers to the function as a parameter. This function returns a data frame of serial of information depending on the parameter.
See the example below which prints merely the one row.
import pandas as pd ## reading csv file from same directory csvFile = pd.read_csv('myfile.csv') ## creating a DataFrame df = pd.DataFrame(csvFile) ##printing specific row disp(df.loc[[two]]) Output:
~]# python3 eg-five.py No Name Gender 2 3 soro Female person Numpy and CSV
Numpy is another powerful bundle in python that is more often than not used past data scientists and car learning engineers to bargain with big and big data. Numpy in python makes it easier to deal with CSV files. In the following sections we volition see how we can employ numpy to open and read csv files.
Example-1: Opening CSV file using Numpy
numpy.loadtxt() role is used to load data from files. Nosotros can read information from a CSV file using this function and shop information technology in a Numpy assortment.. Come across the instance below which provides the bones syntax to open a CSV file using Numpy.
import numpy as np ## opening csv file using numpy data = np.loadtxt("myfile.csv", dtype=str) impress(data) Output:
~]# python3 eg-1.py [['No,' 'Proper name,' 'Gender'] ['i,' 'Erlan,' 'Male person'] ['2,' 'Alex,' 'Male person'] ['iii,' 'soro,' 'Female'] ['4,' 'Khan,' 'Male']] In the to a higher place case, you can see that we take explicitly defined data type to string because our CSV file contains data in cord form. If we will not provide the dtype explicitly, the by default numpy will treat the data every bit float and nosotros will get an error.
Example-2: Python module to read CSV file to Numpy assortment
We can read csv files using the CSV module in python which stores data in a list. Then nosotros tin can convert this list into a numpy assortment and perform the useful features that numpy provides u.s..
Come across example below to understand how we can read a CSV file in a python module and store data in a numpy assortment.
import csv import numpy as np ## open up csv file from same directory with open up('myfile.csv', 'r') as f: ## read csv file and shop in variable information data = list(csv.reader(f, delimiter=";")) ## information is converted to numpy assortment data = np.array(information) print(data) Output:
~]# python3 eg-2.py [['No, Name, Gender'] ['1, Erlan, Male'] ['ii, Alex, Male'] ['3, soro, Female'] ['iv, Khan, Male person']] How to define delimiter to read CSV file in Python
A delimiter separates columns from each other in a CSV file to form a tabular information set. Common CSV delimiters are space and comma. If we create a csv file or write in a csv file, then we tin use a delimiter to make distinction between columns.
Run into the following example which demonstrate the use of delimiter in Python.
## opens csv file and store information in variable file_object = open up("myfile.csv", "w") ## defining delimiter writer = csv.writer(file_object, delimiter = ",") ## adding information to csv file writer.writerow(["a","b"]) ## close opened csv file file_object.shut() Pandas and CSV delimiter
An optional parameter sep in read_csv() is used to specify the delimiter. The default delimiter in the dataset is comma, that means if we will non specify the delimiter explicitly, python will employ comma as delimiter. Nosotros can specify delimiter other than comma by using sep parameter;
See the example beneath which reads csv file having semicolons every bit delimiter.
df = pd.read_csv("myfile.csv", sep = ';') A vertical bar delimited file can exist read past:
df = pd.read_csv("myfile.csv", sep = '|') Summary
In this article, nosotros learned about opening, reading and writing in a CSV file using Python. Nosotros as well learned about python modules which are used to write, read and open CSV files. Moreover, we embrace some useful Python packages including pandas and numpy and learn some of the useful features that these packages provide. This article also provides examples forth with explanation and give full agreement of CSV files in python
Further Readings
Python CSV
Pandas CSV
Use Numpy to read CSV
Source: https://www.golinuxcloud.com/python-read-write-csv/
Post a Comment for "How to Read Csv File With Different Number of Columns Python"