Yes I agree. There are some things you can do for example you can have client certificates, only trusted clients can talk to your ci, and onlt trusted clients can push images. Only dev with ssh keys can push code.

Your ci and registry are locked down (a container).

All that might help.