np.exp turns a large input into inf and only warns instead of raising an error. The overflow matters because it turns a softmax into NaNs, and the fix is one subtraction.
What NumPy exp computes
np.exp is a universal function, or ufunc, so it applies the same operation to every element of its input without a Python loop. Each element comes back as e raised to that value, where e is Euler’s number and equals roughly 2.718281828.
import numpy as np
a = 6
print("np.exp(a) =", np.exp(a))
print("returned type:", type(np.exp(a)).__name__)
np.exp(a) = 403.4287934927351
returned type: float64
Six is small enough to check by hand, and e raised to the sixth is 403.4287934927351. The input was a Python int while the return value is a float64, which is the first hint that NumPy treats this as a floating-point operation from the start.
Negative inputs give the reciprocal of the positive version.
import numpy as np
a = -6
print("np.exp(-6) =", np.exp(a))
print("1 / np.exp(6) =", 1 / np.exp(6))
print("the two are equal:", np.exp(-6) == 1 / np.exp(6))
np.exp(-6) = 0.0024787521766663585
1 / np.exp(6) = 0.0024787521766663585
the two are equal: True
Writing the call with the double-asterisk operator instead gives an answer that differs in the final bit.
import numpy as np
import math
x = 2.5
print("np.exp(x) :", np.exp(x))
print("math.exp(x) :", math.exp(x))
print("np.e ** x :", np.e ** x)
print("all identical :", np.exp(x) == math.exp(x) == np.e ** x)
np.exp(x) : 12.182493960703473
math.exp(x) : 12.182493960703473
np.e ** x : 12.182493960703471
all identical : False
np.exp and math.exp agree exactly at 2.5 while the double-asterisk form is one unit in the last place away. That difference is invisible in a print statement and enough to break an equality check against a literal.
The signature and the arguments worth knowing
The full signature is long, and only two of its arguments change the result rather than the storage. The slash marks x as positional-only, so every option after it must be passed by keyword.
import numpy as np
print(np.exp.__doc__.strip().splitlines()[0])
exp(x, /, out=None, *, where=True, casting='same_kind', order='K', dtype=None, subok=True[, signature])
| Parameter | Purpose | Required |
|---|---|---|
| x | Input array or scalar. The only positional argument. | Yes |
| out | An existing array that receives the result instead of a new allocation. | No |
| where | A boolean mask. Only the True positions are written. | No |
| dtype | Overrides the result dtype that is inferred from the input. | No |
| casting, order, subok | Buffer and memory-layout controls that rarely matter outside tight loops. | No |
The out and where arguments work as a pair. Passing out lets NumPy write into memory you already own, and where selects which slots receive the computed value.
import numpy as np
x = np.array([1.0, 2.0, 3.0, 4.0])
out = np.zeros(4)
np.exp(x, out=out, where=np.array([True, False, True, False]))
print("input :", x)
print("out :", out)
input : [1. 2. 3. 4.]
out : [ 2.71828183 0. 20.08553692 0. ]
The mask is False at index 1 and 3, so those slots keep the zeros that were already in out and only the other two get overwritten. Leave out as None and NumPy allocates a fresh array instead, which means the skipped positions hold whatever was in that memory rather than zero.
Scalars, lists, and arrays
A plain Python list works as input because NumPy converts it to an array before the ufunc runs, and the return value is always an array.
import numpy as np
a = [0, 3, -2, 1]
ans = np.exp(a)
print("input :", a)
print("output :", ans)
print("type :", type(ans).__name__, ans.dtype)
input : [0, 3, -2, 1]
output : [ 1. 20.08553692 0.13533528 2.71828183]
type : ndarray float64
Each element is handled on its own, so the shape of the input decides the shape of the output and nothing else changes.
import numpy as np
a = [[2, -4, 1],
[0, 1, 5]]
ans = np.exp(a)
print(ans)
print("dtype:", ans.dtype)
[[7.38905610e+00 1.83156389e-02 2.71828183e+00]
[1.00000000e+00 2.71828183e+00 1.48413159e+02]]
dtype: float64
A two-dimensional input keeps its two rows and three columns, and higher dimensions behave the same way without any reshaping on your side.
import numpy as np
a = np.arange(8, dtype=float).reshape(2, 2, 2) - 4
print("shape in :", a.shape)
out = np.exp(a)
print("shape out:", out.shape)
print(out)
shape in : (2, 2, 2)
shape out: (2, 2, 2)
[[[1.83156389e-02 4.97870684e-02]
[1.35335283e-01 3.67879441e-01]]
[[1.00000000e+00 2.71828183e+00]
[7.38905610e+00 2.00855369e+01]]]
Integer arrays come back as float64, because the exponential of most integers is not an integer.
import numpy as np
ints = np.array([1, 2, 3], dtype=np.int64)
print("input dtype :", ints.dtype)
print("output dtype:", np.exp(ints).dtype)
print("output :", np.exp(ints))
input dtype : int64
output dtype: float64
output : [ 2.71828183 7.3890561 20.08553692]
The input array stays int64 in memory while the result is float64, because NumPy chooses the output dtype instead of inheriting the input’s.
import numpy as np
import math
import time
n = 1_000_000
values = np.linspace(-5, 5, n)
t0 = time.perf_counter()
np_out = np.exp(values)
t1 = time.perf_counter()
py_out = [math.exp(v) for v in values.tolist()]
t2 = time.perf_counter()
print(f"np.exp : {t1 - t0:.4f} s")
print(f"math.exp : {t2 - t1:.4f} s")
print(f"ratio : {(t2 - t1) / (t1 - t0):.1f}x")
print("largest gap:", abs(np.array(py_out) - np_out).max())
np.exp : 0.0082 s
math.exp : 0.1327 s
ratio : 16.2x
largest gap: 0.0
Calling np.exp on a whole array rather than looping over math.exp is a throughput decision. Across a million values, np.exp finished in 0.0082 seconds and a list comprehension calling math.exp took 0.1327 seconds, roughly 16 times slower.
Both approaches produced identical values, so the difference is speed rather than accuracy.
The largest input NumPy exp can take
float64 tops out near 1.7977e308, and e raised to any exponent above about 709.78 is larger than that number. Two printed lines show the transition.
import numpy as np
print("exp(709.0) =", np.exp(709.0))
print("exp(710.0) =", np.exp(710.0))
np.exp(709.0) returns 8.218407461554972e+307, which is still representable. One step further, np.exp(710.0) returns inf and NumPy writes RuntimeWarning: overflow encountered in exp to standard error.
Overflow does not raise an exception, so a loop that averages an array of exponentials keeps running and the damage spreads to whatever consumes the result.
import numpy as np
for dt in (np.float64, np.float32):
finfo = np.finfo(dt)
print(dt.__name__, "| max:", finfo.max, "| ln(max):", np.log(finfo.max), "| smallest subnormal:", finfo.smallest_subnormal)
float64 | max: 1.7976931348623157e+308 | ln(max): 709.782712893384 | smallest subnormal: 5e-324
float32 | max: 3.4028235e+38 | ln(max): 88.72284 | smallest subnormal: 1e-45
The two precisions stop at very different places, and the dividing line is the natural logarithm of the largest finite value.
| dtype | Largest finite value | First exponent that overflows | Smallest subnormal |
|---|---|---|---|
| float64 | 1.7976931348623157e+308 | 710.0 | 5e-324 |
| float32 | 3.4028235e+38 | 90.0 in practice, 88.72 at the limit | 1e-45 |
A dtype of float32 overflows at an input of 90, while the same value is unremarkable in float64.
import numpy as np
print("float64 max exp:", np.exp(np.float64(709.78)))
print("float32 max exp:", np.exp(np.float32(88.72)))
big = np.array([90.0], dtype=np.float32)
print("float32 90.0 :", np.exp(big))
float64 max exp: 1.7928227943945155e+308
float32 max exp: 3.3931806e+38
float32 90.0 : [inf]
A single array can also hold finite results and infinities at the same time, so one large entry does not stop the rest from computing.
import numpy as np
x = np.array([700.0, 710.0, 1000.0])
print("input :", x)
print("output:", np.exp(x))
input : [ 700. 710. 1000.]
output: [1.01423205e+304 inf inf]
Inputs below about -745 underflow in the same way.
import numpy as np
print("exp(-745.0) =", np.exp(-745.0))
print("exp(-1000.0) =", np.exp(-1000.0))
exp(-745.0) = 5e-324
exp(-1000.0) = 0.0
exp(-745.0) returns the smallest subnormal float64 rather than zero, while exp(-1000.0) returns a plain 0.0.
A value sitting at that floor has no useful reciprocal, so the infinity turns up in the next calculation instead of this one.
Keeping a softmax stable
The log-sum-exp shift is the standard fix for that ceiling, and softmax is the cleanest place to watch it work.
A softmax exponentiates every logit and then divides by their sum. Logits near 1000 overflow inside the exponential, so the division never gets a usable numerator.
import numpy as np
def softmax_naive(logits):
e = np.exp(logits)
return e / e.sum()
def softmax_stable(logits):
e = np.exp(logits - logits.max())
return e / e.sum()
logits = np.array([1000.0, 1001.0, 1002.0])
print("naive :", softmax_naive(logits))
print("stable:", softmax_stable(logits))
The direct version returns nan for all three entries, because an infinity divided by an infinity is undefined.
NumPy reports the two failures separately. The overflow inside exp comes first, and the invalid value inside the division follows it.
Subtracting the largest logit first keeps every exponent at or below zero, which caps the exponentials at 1 and leaves the largest one exactly at 1. The ratio between the entries does not change, because the same factor of e raised to the negative maximum appears in both the numerator and the denominator and cancels.
On small logits the two implementations agree to within 2.8e-17, so the shift costs nothing when it is not needed.
import numpy as np
logits = np.array([2.0, 1.0, 0.1])
naive = np.exp(logits) / np.exp(logits).sum()
shifted = logits - logits.max()
stable = np.exp(shifted) / np.exp(shifted).sum()
print("naive :", naive)
print("stable:", stable)
print("max abs diff:", np.abs(naive - stable).max())
naive : [0.65900114 0.24243297 0.09856589]
stable: [0.65900114 0.24243297 0.09856589]
max abs diff: 2.7755575615628914e-17
More precision near zero with expm1
Very small inputs lose precision long before they overflow, and NumPy ships a companion function for that case. np.expm1 computes e raised to x minus one in a single step, which avoids the cancellation that comes from subtracting a near-one exponential from one.
import numpy as np
tiny = 1e-12
print("exp(tiny) - 1 :", np.exp(tiny) - 1)
print("expm1(tiny) :", np.expm1(tiny))
print("expm1(-1e-9) :", np.expm1(-1e-9))
exp(tiny) - 1 : 1.000088900582341e-12
expm1(tiny) : 1.0000000000005e-12
expm1(-1e-9) : -9.999999995e-10
For an input of 1e-12, subtracting one from np.exp(1e-12) gives 1.000088900582341e-12 while np.expm1(1e-12) gives 1.0000000000005e-12. The second value is the correct one, and the first is wrong from the fifth digit onward because 1e-12 disappears into the leading 1.0 of the exponential.
Reach for expm1 whenever the result is going to be added to, or compared against, a number close to 1.
import numpy as np
x = np.array([0, 1, 2, 3])
print("np.exp2(x) :", np.exp2(x))
print("2 ** x :", 2 ** x)
print("identical :", np.array_equal(np.exp2(x), 2 ** x))
np.exp2(x) : [1. 2. 4. 8.]
2 ** x : [1 2 4 8]
identical : True
The same family includes np.exp2, which computes 2 raised to each element and matches the double-asterisk operator on integers while returning floats.
Plotting the exponential curve
Plotting e raised to x over a few units shows why the numbers behave the way they do. Between x = -2 and x = 0 the curve is nearly flat and never reaches 1.
import numpy as np
import matplotlib.pyplot as plt
x = np.linspace(-2, 3, 200)
y = np.exp(x)
fig, ax = plt.subplots(figsize=(9, 5.5))
ax.plot(x, y, color="#AA5800", linewidth=2.6, label="y = e^x")
marks = np.array([-2, -1, 0, 1, 2, 3], dtype=float)
ax.plot(marks, np.exp(marks), "o", color="#4555A5", markersize=7)
for m in marks:
ax.annotate(f"({m:.0f}, {np.exp(m):.2f})",
(m, np.exp(m)), textcoords="offset points", xytext=(7, 9), fontsize=9)
ax.axhline(1.0, color="#DDE7EC", linewidth=1.2, zorder=0)
ax.axvline(0.0, color="#DDE7EC", linewidth=1.2, zorder=0)
ax.set_xlim(-2.35, 3.55)
ax.set_ylim(-1.2, 23.5)
ax.set_xlabel("x")
ax.set_ylabel("e^x")
ax.set_title("y = e^x across x = -2 to 3")
ax.grid(alpha=0.25)
ax.legend(loc="upper left")
fig.tight_layout()
fig.savefig("plot_exp.png", dpi=150, facecolor="white")
print("saved plot_exp.png")
The same function reaches 20.09 by x = 3, and each extra unit adds more than the one before it. Extending that curve to x = 710 is what produces the infinity from earlier, and no amount of plotting resolution changes where the ceiling sits.
Common questions about NumPy exp
The answers below cover the details that matter once the overflow boundary is clear.
Does np.exp work on a Python list?
Yes. NumPy converts the list to an array before the ufunc runs and returns a float64 array, so a list of ints comes back as an array of floats with the same length.
Why does np.exp return inf instead of raising an error?
Floating-point overflow saturates at infinity rather than raising a Python exception, and NumPy reports it as a RuntimeWarning. Wrap the call in np.errstate with over set to raise when you want a FloatingPointError, or set it to ignore when the infinity is expected.
When should you use expm1 instead of exp?
Use expm1 when the input is near zero and you need e raised to x minus one. Computing exp first and subtracting one throws away the leading digits, so the single-step version keeps precision that the two-step version loses.
Does np.exp preserve the shape of its input?
It does. The ufunc applies element-wise after broadcasting, so a two by three input returns a two by three array, a higher-dimensional input keeps every axis, and a scalar returns a NumPy scalar.
How do you stop np.exp from overflowing inside a softmax?
Subtract the largest logit before exponentiating. The shift cancels between the numerator and the denominator, so the probabilities are unchanged while the largest exponent becomes exactly 1 and nothing overflows.
