Caps are not always necessary. unless metal could come into contact with them. most batterys do not have caps on them. and plastic glue will work(assuming they are plastic.

You dont glue it

You can just add some honey and vinegar

battery cap glue. WTF?