I think this should be true. Here's my reasoning:
To begin with, let's formalize the concept of the number of repeating decimals. If you've done this kind of long division by hand then you will know that the decimals repeat when we get a remainder of $1$ on some iteration. This essentially means that if $\frac{1}{p}$ has $r$ repeating digits, then $10^r \equiv 1 \pmod{p}.$ To be more precise, we should have that the number of repeating decimals is the smallest such $r$ which satisfies this, because if there was a smaller number then the digits would repeat then instead. Now that we have a uniquely defined value for the number of repeating digits (supposing the digits repeat, which is true for prime numbers greater than $5$) let this be represented by the function $r(n)$ from now on.
Now for our prime number $p$ suppose that we have $r(p) = m \cdot r_x,$ where $m$ and $r_x$ are integers and $r_x \neq r(p).$ We can do this whenever $r(p)$ is not $1,$ which it will not be for any prime number $p > 5.$ Because $r(p)$ must be the smallest value $r$ such that $10^r \equiv 1 \pmod{p},$ we must have that $10^{r_x} \not\equiv 1,$ or equivalently that $10^{r_x} - 1 \not\equiv 0.$ Because $p$ is prime, this is equivalent to saying that $10^{r_x} - 1$ is coprime with $p.$
Now, by consequence of Euclid's lemma, we can say that $10^{r(p)} - 1 \equiv 0 \pmod{p}$ implies that $\frac{10^{r(p)} - 1}{10^{r_x} - 1} \equiv 0 \pmod{p}.$ To be clear this only holds if the left-hand side is still an integer, but we know this to be the case because we can apply a famous identity for geometric sums, $\sum_{k = 0}^n r^k = \frac{r^{n+1} - 1}{r - 1}.$ So, we can rewrite our result as $\sum_{k = 0}^{m-1} 10^{kr_x} \equiv 0 \pmod{p},$ which is to say that the sum is divisible by $p.$
This may seem like a fairly random statement, but believe it or not this is actually pretty much what we've set out to prove. To see why, think a bit more in-depth about what you're doing when you concatenate your number $n$ with itself $m$ times. You're really starting with your number $n,$ then adding a copy of the number which has been shifted $r_x$ places to the left, which is to say you add $n \cdot 10^{r_x}.$ Then you repeat this process of multiplying $10^{r_x}$ and adding the copies on until you have $r(p)$ digits, which means you have to make $m$ copies with $r_x$ digits each. What you should get at the end would be $N = \sum_{k = 0}^{m - 1} n \cdot 10^{kr_x},$ noting that the upper bound is $m-1$ because $k$ starts at $0.$ Now, because $n$ is a constant with respect to the sum, we can simply pull out the $n$ to get $N = n \cdot \sum_{k = 0}^{m - 1} 10^{kr_x}.$ And finally, because we showed before that $\sum_{k = 0}^{m - 1} 10^{kr_x}$ is divisible by $p, N$ should also be divisible by $p$ regardless of our choice of $n.$
Sorry if I lost you in there, I know you said you're not a mathematician so I'd understand if that was a bit hard to follow. In any case, thank you for the question, I had a lot of fun figuring this out!
Edit: a while after writing this I thought of an interesting addendum which I wanted to share. The base $10$ is not important to the meat of this argument, so it should work for a number system with an arbitrary base, so long as the appropriate restrictions are made on the primes you use.
As an example, consider $\frac{1}{11_{10}}$ in binary, which is represented as $0.\overline{0001011101}_2.$ This has $10$ repeating digits, so our eligible factors are $1,2,$ and $5.$ Checking the corresponding sums gives us $1111111111_2 = 1023_{10}, 101010101_2 = 341_{10},$ and $100001_2 = 33_{10},$ all of which are divisible by $11_{10}.$