How To Open, Read, Write Files with Python

Files are an essential part of working with computers, thus using Python to write to and read from a file are basic skills that you need to master. In this afrticle, I’ll show you how to do the things you came here for, e.g.:

  • How to open a file in Python
  • Reading a file with Python (both at once or line-by-line)
  • Writing to a file with Python
  • Copy, move, rename, and delete files

When working with files, there will come that point where you need to know about file modes and permissions. I included an explanation on this topic as well, in case you need it.

Open a file in Python

In Python, we open a file with the open() function. It’s part of Python’s built-in functions, you don’t need to import anything to use open(). The open() function expects at least one argument: the file name. If the file was successfully opened, it returns a file object that you can use to read from and write to that file.

As soon as you open a file with Python, you are using system resources that you need to free once you’re done. If you don’t, you create a so-called resource leak. If you do this often enough, you might run into problems, because there’s a limit to the number of open files a regular operating system user can have.

You could decide to skip closing the file because once your program finishes and the Python interpreter closes, resources are freed up automatically. That’s nice and considerate of the Python devs, but a lot of software keeps running indefinitely. With such software, you need to be more careful and close the resources you don’t need anymore. For this reason, it’s a good habit to close a file when you’re done with it.

Closing a file used to be a manual task, that can be forgotten easily. Luckily, Python has the ‘with’ statement to help us out.

The old fashioned way to open a file

To illustrate why the with statement is so useful, let’s first open and close a file in the traditional, manual way:

The old-fashioned way to open a file. You should prefer using the with-statement.

If you’re feeling adventurous, try and remove the f.close(). It should work the same; Python won’t complain and will close the file resource on exit.

If the interactive example above doesn’t work for some reason, here’s a text version of the same code:

f = open('text.txt')
print(f.read())
f.close()

Read file content at once

In the example above, you can see how we read all contents from the file so we can print it. If you want to read all the file content into a single string, at once, use the read() method without arguments. This is no problem for small files, but do realize that you load all the contents into memory at once. This could be a problem with large files, but we’ll get to that soon.

Open a file with the with-statement

There’s a potential problem in the previous example. If an exception occurs, the f.close() call will never be reached. And this is a trivial example, but you can imagine that with more complicated software, in which exceptions might get caught so the software can keep running, it can go wrong quickly.

Another problem that may arise, is that you simply forget to write the close() call. It happens, we are all humans. Therefore, with modern Python, the following method is the recommended way:

If the interactive example above doesn’t work for some reason, here’s a text version of the same code:

with open('text.txt') as f:
    text = f.read()

print(text)

In this example, the with-statement makes sure the file is closed, even if an exception occurs. That’s because Python closes the resource automatically as soon as we step out of the scope of this with-statement.

Due to this automatic close, you can not use the file (called f in our case) outside of the with-statement. If you want to use it again, you have to open it again.

File modes

So far, we used the open() function with just one argument: the file name. It’s a mandatory argument, but open() also has a number of optional extra arguments, like the file mode. This mode defaults to ‘rt’, which stands for ‘read text’. You’re opening the file for reading, which means you can not write to it, and you’re expecting it to be a text file.

Other modes that you can pass are listed in the following table:

CharacterMeaning
ropen for reading (default)
wopen for writing, truncating the file first
xcreate a new file and open it for writing
aOpen for writing, append to the end of the file if it exists already
tText mode (the default), can be used in combination with rwxa
bBinary mode (as opposed to text mode), can be used in combination with rwxa
+open a disk file for updating (reading and writing)
File modes you can use for open()’s mode argument

I won’t go into the details of all the modes here but instead, I’ll explain and demonstrate most of them in the appropriate sections below.

Python write to file

Now that you know about file modes, let’s start by writing some text to a file in Python. We can do so in three steps:

  1. First, we need to determine the file mode. If you look at the table above, we’ll need to use ‘w’ and ‘t’. Since ‘t’ is the default, we can leave it out.
  2. Next, we need to open the file
  3. And finally, we call the write() method on our file object.

Write expects one argument in this case: a Python string. If you have data of another type, like a number, you need to convert it to a string manually. After all, text files don’t have the concept of data types: they simply contain text.

In the example below, we convert integers to strings using the str() function, and write them to the file:

Writing a file in Python, using the write mode

If the interactive example above doesn’t work, here’s the same code:

with open('test.txt', 'w') as f:
    for i in range(1, 5):
        f.write(str(i))

with open('test.txt', 'r') as f:
    print(f.read())

There is now a file on your file system (in the same directory as your script), with the following content:

1234

Newlines

Did you expect each number to be on a new line?

When working with files, you need to explicitly specify newlines. You can try this in the interactive example above. Adapt the code to make it look like this:

with open('test.txt', 'w') as f:
    for i in range(1, 5):
        f.write(f'Number {i}\n')

with open('test.txt', 'r') as f:
    print(f.read())

Note that I used an f-string here. I personally like them a lot, but you can also use something like str(i) + '\n'.

Append to a file

Now to we have a test file, we can also append some extra data to it. By default, when we open a file for writing, we overwrite anything that’s already present. So, again, we first look in the table above which mode we can use to append data to our file. We need mode ‘a’, for append.

In the following example, we:

  1. Write to a file just like in the previous example.
  2. After that, we open the file again, this time with the append flag, and add some extra lines.
  3. Read back the file contents and print to screen so we can inspect the result

Here you go:

Append text to a file in Python

If the interactive example above doesn’t work, here’s the same code:

# First create a file, with a couple of lines
with open('test.txt', 'w') as f:
    for i in range(1, 5):
        f.write(f'Number {i}\n')

# Now add some extra lines using append mode
with open('test.txt', 'a') as f:
    for i in range(5, 8):
        f.write(f'Append number {i}\n')

with open('test.txt') as f:
    print(f.read())

Python read file to list

With small files, it can be convenient to read all lines at once into a list. There are two ways to do this:

with open('test.txt') as f:
    lines = list(f)

# lines = ['1\n', '2\n', '3\n', '4\n', '5\n', '6\n', '7\n']

Is equivalent to:

with open('test.txt') as f:
    lines = f.readlines()

# lines = ['1\n', '2\n', '3\n', '4\n', '5\n', '6\n', '7\n']

Read file line-by-line

You don’t always have the luxury of reading files at once. For instance, when you need to work with 140 GB-sized log files, you can’t read it all into memory and get away with it. Your program will likely crash, or at least become extremely slow.

In such cases, it is much more efficient to parse the file line-by-line. To read a file line by line, we can treat the opened file as an iterator. After opening the file, we can use a Python for-loop to iterate over the lines:

with open('myfile.txt', 'r') as f:
    for line in f:
        print(line)

Common Python file operations

Python has built-in modules to perform common file operations, like deleting files, creating directories, moving files, and so on. Most of these functions are available in the `os` module. In this section, we’ll visit some of the operating you might need to perform.

Check if file or directory exists

To check if a file exists, we can use the isfile function:

import os
os.path.isfile('myfile.txt')

This will return True if the file exists, and False if it doesn’t.

Similarly, to check if a directory exists, we can use the isdir function:

import os
os.path.isdir('mydir')

Create a directory

To create a directory, use the mkdir function from the os module:

import os
os.mkdir('mydir')

Delete a file

To delete a file, we can use the following command:

import os
os.remove('myfile.txt')


This will delete the file. If the file doesn’t exist, it will raise an exception.

Rename (or move) a file

To rename or move a file, we can use the following command:

import os
os.rename('myfile.txt', 'myfile_renamed.txt')

A rename operation works as long as the file stays on the same file system. If you want to move a file from one file system to another, you need the higher-level move function from shutil, as explained below.

Move a file with shutil

To move a file in a way that resembles the mv command on the terminal shell, we can use the shutil module. An important distinction from os.renae is that this function can move files between file systems as well:

import shutil

shutil.move('/mnt/filesystem1/myfile.txt', '/mnt/filesystem2/mydir')

# Move to a directory, keeping the name intact
shutil.move('/home/erik/myfile.txt', '/home/erik/backups/')

Copy files with Python

The shutil module is a more high-level module, and provides a number of functions that can be used to copy single files or copy and remove entire trees of files:

import shutil
# Copy a single file
shutil.copy('/home/erik/myfile.txt', '/home/erik/myfile_copy.txt')
# Copy entire tree of files
shutil.copytree('mydir', 'mydir_copy')
# Remove a tree of files
shutil.rmtree('mydir')

Handling Python file exceptions

If an error occurs, the shutil and os modules will raise an exception. You can use the try and except blocks to handle the exception. The exception you need to handle is almost always of type OSError. However, sometimes you may need to handle other exceptions as well, like SameFileError.

I’ve written a detailed tutorial on how to work with exceptions and try-catch blocks that you might want to read.

About Unix file permissions

When working with files, you’ll inevitably run into file permissions. Consider this a short primer on Unix file permissions. Chances are you’ll run into them sooner than later, especially when working in the cloud since most cloud servers are running Linux.

Permissions are specified per file. When you list a directory, you’ll be able to see its permissions. For example, under Linux, we can do so with the ls -l command:

$ ls -l
total 0
-rwxr-xr-x 1 erik erik 0 Jun 25 10:33 build_script.sh
-rw-r--r-- 1 erik erik 0 Jun 25 10:33 myapp.py

In the output, you see two dummy files I created for demonstration purposes. The first is a script file, and the second is a Python file called myapp.py. At the start of the line, you see cryptic letters like r, w, and x that define the file permissions. Files can have the following permissions:

LetterPermissions
rThe file can be read
wThe file is writeable
xThe file is executable
Unset, permission not granted
Possible file permissions

About users and groups

Knowing what permissions there are is not enough, though. As you can see in the example listing above, the r gets repeated three times for each file. If you were wondering why: it’s because computer systems usually have multiple users and groups.

Each file has permissions for three access types:

  • Everybody else, also called ‘world’
  • group: a group of 0 or more users
  • user: the owner of the file. The one who creates a file, automatically becomes the owner

The permissions are listed in this order as well. So we have three permissions fields for each access type, resulting in a nine-character long string. The following ASCII art should clear things up:

    World  Group   User
0   123    456     789
-   rwx    r-x     r-x

Positions 1, 2, 3 define permissions for the entire world
Positions 4, 5, 6 define permissions for the group that the user is in
Positions 7, 8, 9 define permissions for the user
Position 0 is the file type (see below)

About file types

Finally, there are several file types. The most well-known are regular files and directories, but there are more types. This is the complete list:

  •  : regular file
  • d : directory
  • c : character device file
  • b : block device file
  • s : local socket file
  • p : named pipe
  • l : symbolic link

To demonstrate, you can inspect the file /dev/random:

ls -l /dev/random
crw-rw-rw- 1 root root 1, 8 jun 22 08:44 /dev/random

It’s a character device that offers a stream of random data when read from. As you can see, it’s writable as well. In this case, writing to /dev/random will update the entropy pool.

There’s a lot more to learn, and you might also like to dive into our section on using the Unix shell. So go and explore if you want or need to, but this quick introduction here should get you started.

Keep learning

Use these resources to expand your knowledge and understanding:

About Erik van Baaren

Erik is the owner of Python Land and the author of many of the articles and tutorials on this website. He's been working as a professional software developer for 25 years, and he holds a Master of Science degree in computer science. His favorite language of choice: Python! Writing good articles takes time and effort. Did you like this tutorial? You can buy him a coffee to show your appreciation.

Leave a Comment