How to Handle the MemoryError in Python (2023)

A MemoryError means that the interpreter has run out of memory to allocate to your Python program. This may be due to an issue in the setup of the Python environment or it may be a concern with the code itself loading too much data at the same time.


An Example of MemoryError

To have a look at this error in action, let’s start with a particularly greedy piece of code. In the code below, we start with an empty array and use nested arrays to add strings to it. In this case, we use three levels of nested arrays, each with a thousand iterations. This means at the end of the program, the array s has 1,000,000,000 copies of the string "More."

s = []for i in range(1000): for j in range(1000): for k in range(1000): s.append("More")


As you might expect, these million strings are a bit much for, let’s say, a laptop to handle. The following error is printed out:

C:\code\Python\MemErr\venv\3K\Scripts\python.exe C:/code/python/MemErr/main.pyTraceback (most recent call last): File "C:/code/python/MemErr/", line 6, in <module> s.append("More")MemoryError

In this case, the traceback is relatively simple as there are no libraries involved in this short program. After the traceback showing the exact function call which caused the issue, we see the simple but direct MemoryError.


Two Ways to Handle A MemoryError in Python

Appropriate Python Set-up

This simplest but possibly least intuitive solution to a MemoryError actually has to do with a potential issue with your Python setup. In the event that you have installed the 32-bit version of Python on a 64-bit system, you will have extremely limited access to the system's memory. This restricted access may cause MemoryErrors on programs that your computer would normally be able to handle.

Attention to Large Nested Loops

If your installation of Python is correct and these issues still persist, it may be time to revisit your code. Unfortunately, there is no cut and dry way to entirely remove this error outside of evaluating and optimizing your code. Like in the example above, pay special attention to any large or nested loops, along with any time you are loading large datasets into your program in one fell swoop.

In these cases, the best practice is often to break the work into batches, allowing the memory to be freed in between calls. As an example, in the code below, we have broken out earlier nested loops into 3 separate loops, each running for 333,333,333 iterations. This program still goes through one million iterations but, as the memory can be cleared through the process using a garbage collection library, it no longer causes a MemoryError.

An Example of Batching Nested Loops

import gcs = []t = []u = []for i in range(333333333): s.append("More")gc.collect()for j in range(333333333): t.append("More")gc.collect()for k in range(333333334): u.append("More")gc.collect()


How to Avoid a MemoryError in Python

Python’s garbage collection makes it so that you should never encounter issues in which your RAM is full. As such, MemoryErrors are often indicators of a deeper issue with your code base. If this is happening, it may be an indication that more code optimization or batch processing techniques are required. Thankfully, these steps will often show immediate results and, in addition to avoiding this error, will also vastly shorten the programs' runtime and resource requirements.


Track, Analyze and Manage Errors With Rollbar

Managing errors and exceptions in your code is challenging. It can make deploying production code an unnerving experience. Being able to track, analyze, and manage errors in real-time can help you proceed with more confidence. Rollbar automates error monitoring and triaging, making fixing Python errors easier than ever. Sign Up Today!


How do I overcome MemoryError in Python? ›

To fix this error, you can reduce the size of your dataset, use chunking, use Dask, or use a larger machine with more memory. By following these solutions, you can ensure that your data analysis projects run smoothly and efficiently, even with very large datasets.

How do we manage memory in Python? ›

We use the Heap for implement dynamic memory management. We can use the memory throughout the program. As we know, everything in Python is an object means dynamic memory allocation inspires the Python memory management. Python memory manager automatically vanishes when the object is no longer in use.

How do I clear memory in Python code? ›

The del keyword is used to delete the objects in Python. This function is primarily used for memory management, as it frees up memory that is no longer being used by the program. By deleting unnecessary objects, you can reduce the amount of memory that your program is using, which can help improve its performance.

How do I limit memory usage in Python? ›

  1. sample 1. import standard memory_limit = 256M.
  2. sample 2. from memory import config memory_limit = 256M.
  3. sample 3. from config import memory memory_limit = 256M.
16 Apr 2022

Can you manually manage memory in Python? ›

In languages like C or Rust, memory management is the responsibility of the programmer. The programmer has to manually allocate memory before it can be used by the program and release it when the program no longer needs it. In Python, memory management is automatic!

How do I give Python more memory? ›

Python doesn't limit memory usage on your program. It will allocate as much memory as your program needs until your computer is out of memory. The most you can do is reduce the limit to a fixed upper cap. That can be done with the resource module, but it isn't what you're looking for.

How is memory managed in Python interview questions? ›

12 In Python, how is memory managed? The Python memory is primarily managed by Python private heap space. All Python objects and data structures are located in a private heap. The programmer does not have access to this private heap and interpreter takes care of this Python private heap.

Does Python use a lot of memory? ›

In fact, Python uses more like 35MB of RAM to store these numbers. Why? Because Python integers are objects, and objects have a lot of memory overhead. Let's see what's going on under the hood, and then how using NumPy can get rid of this overhead.

How do I clean up large data in Python? ›

Pythonic Data Cleaning With pandas and NumPy
  1. Dropping Columns in a DataFrame.
  2. Changing the Index of a DataFrame.
  3. Tidying up Fields in the Data.
  4. Combining str Methods with NumPy to Clean Columns.
  5. Cleaning the Entire Dataset Using the applymap Function.
  6. Renaming Columns and Skipping Rows.

How to check memory in Python? ›

Try Python profiler mprof for memory usage

mprof can show you memory usage over the lifetime of your application. This can be useful if you want to see if your memory is getting cleaned up and released periodically.

What happens if you run out of memory in Python? ›

Crashing is just one symptom of running out of memory. Your process might instead just run very slowly, your computer or VM might freeze, or your process might get silently killed. Sometimes if you're lucky you might even get a nice traceback, but then again, you might not.

How do I use less memory in code? ›

  1. Use services sparingly.
  2. Use optimized data containers.
  3. Be careful with code abstractions.
  4. Use lite protobufs for serialized data.
  5. Avoid memory churn.
15 Sept 2023

What is the memory limit of Python list? ›

  • source Therefore the maximum size of a python list on a 32 bit system is 536,870,912 elements. ...
  • A 32-bit process has a theoretical limit of 4 GB of memory, though if your OS is also 32-bit it will obviously be less since the OS will take up some of that memory.
4 Apr 2011

What is the reason of MemoryError in Python? ›

A MemoryError means that the interpreter has run out of memory to allocate to your Python program. This may be due to an issue in the setup of the Python environment or it may be a concern with the code itself loading too much data at the same time.

How do I fix errors in Python? ›

Common Errors in Python and How to Fix Them
  1. Double-check your code for typos or other mistakes before running it.
  2. Use a code editor that supports syntax highlighting to help you catch syntax errors.
  3. Read the error message carefully to determine the location of the error.
13 Mar 2023

How do I exit Python after error? ›

The Many Ways to Exit in Python
  1. raise SystemExit() We can exit from Python code by raising a SystemExit exception: print("Done.") ...
  2. sys. exit() ...
  3. exit() and quit() exit() and quit() are “extra builtins” added by Python's site module. ...
  4. Ctrl-D (from the REPL) Ctrl-D is the universal keyboard shortcut for exit. ...
  5. os. _exit()
10 Oct 2021


Top Articles
Latest Posts
Article information

Author: Eusebia Nader

Last Updated: 13/11/2023

Views: 6480

Rating: 5 / 5 (80 voted)

Reviews: 87% of readers found this page helpful

Author information

Name: Eusebia Nader

Birthday: 1994-11-11

Address: Apt. 721 977 Ebert Meadows, Jereville, GA 73618-6603

Phone: +2316203969400

Job: International Farming Consultant

Hobby: Reading, Photography, Shooting, Singing, Magic, Kayaking, Mushroom hunting

Introduction: My name is Eusebia Nader, I am a encouraging, brainy, lively, nice, famous, healthy, clever person who loves writing and wants to share my knowledge and understanding with you.