There are no constants and no variables. There are only elements in a set.
The epsilon-delta definition begins with "Let epsilon > 0." From that point on, epsilon is a fixed positive real number, i.e. a positive element in R, where R stands for the set of real numbers. Epsilon is definitely NOT an infinitesimal.