Pointers are famously difficult to learn and reason about even though the basic principles are simple. Programming in a style that requires direct manipulation of pointers when it's not actually necessary is usually regarded as unwise because it's so hard to get right.
OP had no problem with pointers prior to trying C++. I think there is a case to be made that C(++) makes pointers unnecessarily confusing and there is no real disconnect between understanding pointers in theory and in practice otherwise
Pointers aren't hard, it's C/C++ that make them complicated. Addresses and indirection in any assembly language are simple and straightforward, easy and even intuitive once you start actually writing programs.
They are though! Indirection in assembly is just something like:
ldr dest, [src, offset]
It's straightforward and pretty hard to mess up, and easy to read to because the format is consistent.
Whereas in C all the following are valid (and it becomes even more confusing with assignment in the declaration statement, tons of footguns and weird syntax):
int* a;
int *a;
int a[];
int a[5];
Assignment is weird too, especially because dereferencing and defining a pointer both use '*'.
*a = c;
a[0] = c;
Then you have structs/unions and their members, and what if those are pointers? You get . and -> syntax. It's weird and complicated, much much more complicated than assembly. That's before you get to casting and types which make C much more complicated than assembly for doing low level stuff.
I used to think I was incapable of learning "real" programming because I didn't get C. When I later read a book on programming in assembly, I realized that everything that had felt so complex was actually not so difficult. C pointer syntax is weird and doesn't parse naturally for many people, especially programming novices who might not yet have a solid grasp on what/how/why they're doing anything.