About a million Rohingyas have fled due to the ethnic cleansing in Myanmar and sought refuge in Bangladesh. The refugees are located in temporary settlements on hilly areas of Cox’s Bazar with inadequate water and sanitation facilities, giving rise to diseases such as cholera, typhoid, and diarrhea. This exploratory study reports drinking water security challenges in two Rohingya refugee camps within the larger camp network—Camp 2 and the recently-built Camp 4 Extension (Camp 4Ext)—to discover the key everyday issues refugees are facing related to drinking water. Both qualitative and quantitative methods have been applied to determining whether contamination is occurring during the collection, transportation, and storage of drinking water by comparing the water quality at the source with that in storage. The results show that Camp 4Ext is more suited for living in several respects compared with the other camps, attributable to significantly better planning during its construction: there is a lower prevalence of diseases, lower water collection times, higher standards of sanitation, and better access to water sources. This study’s outcomes will help camp authorities and the various agencies working there to provide sustainable water and sanitation interventions to improve the wellness of the Rohingya refugees in Cox’s Bazar. The outcomes will also provide useful information and strategic direction to the global scientific and development communities who are working in refugee camps in other parts of the world, to tackle water security challenges.