$\frac{\mathrm d}{\mathrm d z}\mathrm{exp}_0(z) = 1 $

$\mathrm{exp}_{n}(z) = \mathrm{exp}_{n-1}(z) + \dfrac{1}{n!} z^n$

$\frac{\mathrm d}{\mathrm d z}\mathrm{exp}_0(z) = 0 $

$\frac{\mathrm d}{\mathrm d z}\mathrm{exp}_n(z) = \mathrm{exp}_{n-1}(z) = \mathrm{exp}_n(z) - \dfrac{1}{n!} z^n$

Another series which has $a^z$ as limit is the one given below. It has the nice feature that for integers $n<m$, it evaluates to $p(m,n)=a^m$ exactly.

p[m_, z_] := Sum[(a - 1)^k/k! \!\(
\*UnderoverscriptBox[\(\[Product]\), \(j = 0\), \(k - 1\)]\((z - 
       j)\)\), {k, 0, m}] // Expand

Table[Table[p[m, z], {z, 1, m}], {m, 1, 7}]     // 
  Simplify // TableForm

i = 3;
2^i // N
p[5, i] /. {a -> 2} // N

i = 3 + 1/2;
2^i // N
p[5, i] /. {a -> 2} // N

Wikipedia: Exponential function, Matrix exponential, Exponential map

Finite sum over a monoid, Factorial function