Unsure about react-map-gl. Both of those are just RN wrappers around mapbox-gl though, so same functionality should be available. Openstreetmap _directly- (rather than having Mapbox over the top) doesn’t have any particularly great RN wrapper libraries, you can d/l and cache map tiles just fine, but afaik you’d then need to implement that logic yourself.
Note that you need to know exactly what you want implented here: offline maps tens to equal enormous amounts of data, so you need to be careful with this, else it’ll chew through your device storage pretty sharpish