Python Read And Write File: With Examples

Files are an essential part of working with computers, thus using Python to write to and read from a file are basic skills that you need to master. In this article, I’ll show you how to do the things you came here for, e.g.:

• How to open a file in Python
• Reading a file with Python (both at once or line-by-line)
• Writing to a file with Python
• Copy, move, rename, and delete files
• Check if a file or directory exists

When working with files, there will come that point where you need to know about file modes and permissions. I included an explanation on this topic as well, in case you need it.

Open a file in Python

In Python, we open a file with the open() function. It’s part of Python’s built-in functions, you don’t need to import anything to use open(). The open() function expects at least one argument: the file name. If the file was successfully opened, it returns a file object that you can use to read from and write to that file.

As soon as you open a file with Python, you are using system resources that you need to free once you’re done. If you don’t, you create a so-called resource leak. If you do this often enough, you might run into problems, because there’s a limit to the number of open files a regular operating system user can have.

This doesn’t have to be a problem though. When your program exits, all resources are freed up automatically. That’s nice and considerate of the Python devs, but a lot of software keeps running indefinitely. With such software, you need to be more careful and close the resources you don’t need anymore. This way, you make sure other software can access the file safely. Also, when you open a lot of files, each open file takes up memory and other resources. For these reasons, it’s a good habit to close a file when you’re done with it.

Closing a file used to be a manual task, that can be forgotten easily. Luckily, Python has the ‘with’ statement to help us out. We’ll discuss the with statement soon. I first want to show you the ‘old fashioned way of doing things.

The old fashioned way to open a file

To illustrate why the with statement is so useful, let’s first open and close a file in the traditional, manual way:

If you’re feeling adventurous, try and remove the f.close(). It should work the same; Python won’t complain and will close the file resource on exit.

If the interactive example above doesn’t work for some reason, here’s a text version of the same code:

f = open('text.txt')
print(f.read())
f.close()

Python read file content at once

In the example above, you can see how we read all contents from the file so we can print it. If you want to read all the file content into a single string, at once, use the read() method on a file object, without arguments. This is no problem for small files, but do realize that you load all the contents into memory at once. This could be a problem with large files, but we’ll get to that soon.

Using with open() in Python

There’s a potential problem in the previous example. If an exception occurs while Python reads the file, the f.close() call will never be reached. And this is a trivial example, but you can imagine that with more complicated software, in which exceptions might get caught so the software can keep running, it can go wrong quickly.

Another problem that may arise, is that you simply forget to write the close() call. It happens, we are all humans. Therefore, with modern Python, it’s recommended to use the with-statement when you can. By using Python’s with open(), the opened file resource will only be available in the indented block of code:

If the interactive example above doesn’t work for some reason, here’s a text version of the same code:

with open('text.txt') as f:
    text = f.read()

print(text)

In this example, the with-statement makes sure the file is closed, even if an exception occurs. That’s because Python closes the resource automatically as soon as we step out of the scope of this with-statement. Due to this automatic close, you can not use the file (called f in our case) outside of the with-statement. If you want to use it again, you have to open it again.

File modes

So far, we used the open() function with just one argument: the file name. It’s a mandatory argument, but open() also has a number of optional extra arguments, like the file mode. This mode defaults to ‘rt’, which stands for ‘read text’. You’re opening the file for reading, which means you can not write to it, and you’re expecting it to be a text file.

Other modes that you can pass are listed in the following table:

I won’t go into the details of all the modes here but instead, I’ll explain and demonstrate most of them in the appropriate sections below.

Python write file

Now that you know about file modes and how to open a file, we can use Python to write to files as well. We can do so in three steps:

1. First, we need to determine the file mode. If you look at the table above, we’ll need to use ‘w’ and ‘t’. Since ‘t’ is the default, we can leave it out.
2. Next, we need to open the file for writing
3. And finally, we call the write() method on our file object.

Write expects one argument in this case: a Python string. If you have data of another type, like a number, you need to manually convert it to a string. After all, text files don’t have the concept of data types: they simply contain text.

In the example below, we convert integers to strings using the str() function, and write them to the file:

If the interactive example above doesn’t work, here’s the same code:

with open('test.txt', 'w') as f:
    for i in range(1, 5):
        f.write(str(i))

with open('test.txt', 'r') as f:
    print(f.read())

There is now a file on your file system (in the same directory as your script), with the following content:

1234

Newline characters

Did you expect each number to be on a new line?

When writing files, you need to explicitly specify newline characters. You can try this in the interactive example above. Adapt the code to make it look like this:

with open('test.txt', 'w') as f:
    for i in range(1, 5):
        f.write(f'Number {i}\n')

with open('test.txt', 'r') as f:
    print(f.read())

Note that I used an f-string here. I personally like them a lot, but you can also use something like str(i) + '\n'.

Python append to a file

Now that we have a test file, we can also append some extra data. By default, when we open a file for writing, we overwrite anything that’s already present. So, again, we first look in the file modes table above which mode we can use to append data to our file. We need mode ‘a’, for append.

In the following example, we:

1. Write to a file just like in the previous example using Python’s with open().
2. After that, we open the file again, this time with the append flag, and add some extra lines.
3. Read back the file contents and print to screen so we can inspect the result

Here you go:

If the interactive example above doesn’t work, here’s the same code:

# First create a file, with a couple of lines
with open('test.txt', 'w') as f:
    for i in range(1, 5):
        f.write(f'Number {i}\n')

# Now add some extra lines using append mode
with open('test.txt', 'a') as f:
    for i in range(5, 8):
        f.write(f'Append number {i}\n')

with open('test.txt') as f:
    print(f.read())

Python read file to list

With small files, it can be convenient to read all lines at once into a list. There are two ways to do this:

with open('test.txt') as f:
    lines = list(f)

# lines = ['1\n', '2\n', '3\n', '4\n', '5\n', '6\n', '7\n']

Is equivalent to:

with open('test.txt') as f:
    lines = f.readlines()

# lines = ['1\n', '2\n', '3\n', '4\n', '5\n', '6\n', '7\n']

Read file line-by-line

You don’t always have the luxury of reading files at once. For instance, when you need to work with 140 GB-sized log files, you can’t read it all into memory and get away with it. Your program will likely crash, or at least become extremely slow.

It is much more efficient to parse the file line-by-line in such cases. To read a file line by line, we can treat the opened file as an iterator. After opening the file, we can use a Python for-loop to iterate over the lines:

with open('myfile.txt', 'r') as f:
    for line in f:
        print(line)

Common Python file operations

Python has built-in modules to perform common file operations, like deleting files, creating directories, moving files, and so on. Most of these functions are available in the os module, so you’ll need to import it first. In this section, we’ll visit some of the operations you might need to perform.

Use Python to check if file exists

To check if a file exists, we can use the isfile function:

import os

if os.path.isfile('myfile.txt'):
    print("It's a file")

This will return True if the file exists, and False if it doesn’t.

Use Python to check if a directory exists

Similarly, to check if a directory exists, we can use the isdir function:

import os

if os.path.isdir('mydir'):
    print("It's a directory")

If the directory exists, this will return True. Otherwise, it will return False.

Create a directory

To create a directory, use the mkdir function from the os module:

import os

os.mkdir('mydir')

Delete a file

To delete a file, we can use the following command:

import os

os.remove('myfile.txt')

This will delete the file. If the file doesn’t exist, it will raise an exception.

Rename (or move) a file

To rename or move a file, we can use the following command:

import os
os.rename('myfile.txt', 'myfile_renamed.txt')

A rename operation works as long as the file stays on the same file system. If you want to move a file from one file system to another, you need the higher-level move function from shutil, as explained below.

Move a file with shutil

To move a file in a way that resembles the mv command on the terminal shell, we can use the shutil module. An important distinction from os.rename is that this function can move files between file systems as well. Rename can only rename/move a file within the same filesystem:

import shutil

shutil.move('/mnt/filesystem1/myfile.txt', '/mnt/filesystem2/mydir')

# Move to a directory, keeping the name intact
shutil.move('/home/erik/myfile.txt', '/home/erik/backups/')

Copy files with Python

The shutil module is a more high-level module, and provides a number of functions that can be used to copy single files or copy and remove entire trees of files:

import shutil

# Copy a single file
shutil.copy('/home/erik/myfile.txt', '/home/erik/myfile_copy.txt')

# Copy entire tree of files
shutil.copytree('mydir', 'mydir_copy')

# Remove a tree of files
shutil.rmtree('mydir')

Handling Python file exceptions

If an error occurs, the shutil and os modules will raise an exception. You can use the try and except blocks to handle the exception. The exception you need to handle is almost always of type OSError. However, sometimes you may need to handle other exceptions as well, like SameFileError.

I’ve written a detailed tutorial on how to work with exceptions and try-catch blocks that you might want to read.

About Unix file permissions

When working with files, you’ll inevitably run into file permissions. Consider this a short primer on Unix file permissions. Chances are you’ll run into them sooner than later, especially when working in the cloud since most cloud servers are running Linux.

File permissions

Permissions are specified per file. When you list a directory, you’ll be able to see its permissions. For example, under Linux, we can do so with the ls -l command:

$ ls -l
total 0
-rwxr-xr-x 1 erik erik 0 Jun 25 10:33 build_script.sh
-rw-r--r-- 1 erik erik 0 Jun 25 10:33 myapp.py

In the output, you see two dummy files I created for demonstration purposes. The first is a script file, and the second is a Python file called myapp.py. At the start of the line, you see cryptic letters like r, w, and x that define the file permissions. Files can have the following permissions:

Users and groups

Knowing what permissions there are is not enough, though. As you can see in the example listing above, the r gets repeated three times for each file. If you were wondering why: computer systems usually have multiple users and groups.

Each file has permissions for three access types:

• user: the owner of the file. The one who creates a file automatically becomes the owner
• group: a group of 0 or more users
• Everybody else; also called ‘world’

The permissions are listed in this order as well. So we have three permissions fields for each access type, resulting in a nine-character long string. The following ASCII art should clear things up:

    User   Group   World
0   123    456     789
-   rwx    r-x     r-x

Positions 1, 2, 3 define permissions for the user
Positions 4, 5, 6 define permissions for the group that the user is in
Positions 7, 8, 9 define permissions for world
Position 0 is the file type (see below)

File types

Finally, there are several file types. The most well-known are regular files and directories, but there are more types. This is the complete list:

• – : regular file
• d : directory
• c : character device file
• b : block device file
• s : local socket file
• p : named pipe
• l : symbolic link

To demonstrate, you can inspect the file /dev/random:

ls -l /dev/random
crw-rw-rw- 1 root root 1, 8 jun 22 08:44 /dev/random

It’s a character device that offers a stream of random data when read from. As you can see, it’s writable as well. In this case, writing to /dev/random will update the entropy pool.

There’s a lot more to learn, and you might also like to dive into our section on using the Unix shell. So go and explore if you want or need to, but this quick introduction here should get you started.

Keep learning

Use these resources to expand your knowledge and understanding:

• How to load, read, and write YAML in Python
• Working with JSON in Python
• The official Python documentation on working with files
• The official shutil documentation
• Wikipedia page about File permissions
• Our chapter on using the unix shell