Large integer handling in Python (optimization)

I hit the Python int wall on a 10,000-digit number and got a ValueError I did not expect. I had assumed unlimited meant no limits anywhere, which felt safe until str() refused to convert the value. Once I traced the 4300-digit guard and timed a few large ops, the handling rules became concrete.

Why Python ints feel unlimited and where that promise ends

Python ints are arbitrary precision. Under the hood CPython stores them as base 2**30 digits, so 10**100 is a normal int with 101 decimal digits and no overflow like you get in C. I ran 10**100 + 1 == 10**100 and it correctly returned False, because there is no fixed width to wrap around.

The limit you actually hit is string conversion. Since Python 3.11 the interpreter guards int to string with a default of 4300 digits, stored in sys.int_info and adjustable via sys.get_int_max_str_digits(). I tried str(10**10000) and it raised Exceeds the limit, which is the moment most people think ints are broken when they are just being protected.

import sys
print(sys.int_info)
# sys.int_info(bits_per_digit=30, sizeof_digit=4, default_max_str_digits=4300, str_digits_check_threshold=640)
print(sys.get_int_max_str_digits())  # 4300
print(len(str(10**100)))  # 101, well within limit
try:
    print(len(str(10**10000)))
except ValueError as e:
    print(e)  # Exceeds the limit

What you need before you handle big ints safely

You need Python 3.11 or later if you want the guard, plus a plan for when you actually need the decimal string. I tested on Python 3.11.16 with the standard library, so every path below runs without extra installs. Decide first whether you need exact integer math, decimal precision, or fixed-width interop.

import sys
# check once at startup
limit = sys.get_int_max_str_digits()
print(f"current limit {limit}")
# raise only when you control the environment
# sys.set_int_max_str_digits(10000)

How to handle large integers three ways

These three paths cover the cases I actually see, so pick by what you store rather than habit.

Use plain int for exact math

Plain int is the default. It never overflows in the C sense, because it grows as needed, so I keep it for crypto, factorial, and any exact count.

import math

print(2**63 - 1)   # 9223372036854775807
print(2**63)       # 9223372036854775808 still exact
print(2**1000)     # 302 digits, still an int
print(len(str(math.factorial(100))))  # 158 digits
print(math.factorial(100) >> 500)     # shift works at any size

I assumed shifting a huge factorial would be slow. Instead the operation stayed fast because the digit array is compact, so the cost is proportional to digits, not a hidden conversion.

Use Decimal when you need fixed precision

Decimal is for when you need predictable rounding rather than exact integer growth. Set precision once, then divide.

from decimal import Decimal, getcontext

getcontext().prec = 50
print(Decimal(1) / Decimal(7))
# 0.14285714285714285714285714285714285714285714285714
print(1/7)  # 0.14285714285714285 float

getcontext().prec = 10
print(Decimal(1) / Decimal(7))  # shorter, rounded

The float 1/7 looked fine until I compared it to Decimal with prec 50. The float stopped at 17 digits, so for money or repeated rounding I now reach for Decimal instead of scaling ints by hand.

Avoid NumPy ints when you need exactness

NumPy ints are fixed width and they wrap. That makes them fast for arrays but wrong for big-int storage.

import numpy as np

print(np.int64(2**63 -1))  # max int64
try:
    print(np.int64(2**63))   # wraps to negative
except Exception as e:
    print(e)
print(np.int64(2**63 -1) + np.int64(1))  # -9223372036854775808 wrap

# Python int does not wrap
print(2**63)  # 9223372036854775808 exact

I expected NumPy to raise on overflow because Python int does not. Instead it wrapped silently to a negative, which is why I keep NumPy out of exact big-int paths.

Where big ints still break

Three boundaries matter, and each has a different fix.

import sys, time

# 1. String limit
try:
    s=str(10**5000)
except ValueError as e:
    print("blocked:", e)
    sys.set_int_max_str_digits(10000)
    print("after raise, len", len(str(10**5000)))
    sys.set_int_max_str_digits(4300)  # restore

# 2. Performance grows with digits
start=time.time()
x=1
for i in range(5000):
    x = x*2 + 1
    if x.bit_length() > 10000:
        x >>= 5000
print(f"loop time {time.time()-start:.4f}s, bits {x.bit_length()}")

# 3. Float conversion loses
big = 10**16 + 1
print(big)           # exact as int
print(float(big))    # 1e16, lost 1
print(int(float(big)) == big)  # False

• String: guard 4300, raise only when you control the call site
• Memory: big ints grow digit arrays, so avoid holding many huge temporaries
• Speed: ops are O(n) in digits, loop with doubling gets expensive past thousands of digits
• Float: never use float(big) for equality, it rounds at 2**53

What you now have

You have the exact rule and the two guards. I keep ints for exact math, switch to Decimal for fixed precision, and avoid NumPy ints for storage where exactness matters. That split plus the string-limit check has kept big-int code from failing at the conversion step where the ValueError actually lives.

# check I run at startup
import sys
print(sys.get_int_max_str_digits(), sys.int_info.bits_per_digit)

FAQ