The Halting problem is decidable on machines with finite memory. One can enumerate all the states and continually check for a repeated state, albeit being inefficient. (https://en.wikipedia.org/wiki/Halting_problem#Common_pitfalls)
What I struggle to understand is the significance of machines with infinite memory. Why would it be necessary to model a computer having infinite memory? Any program that terminates must have finite memory.
I acknowledge that finding at which point to set the finite bar is a challenge, but I am not convinced that this task is impossible. For instance, for a program with a binary encoding of size $n$, set an upper limit of memory to $2^n$, which is sufficient to encode any possible information contained therein.