While some people swear you shouldn't, others are convinced there's nothing wrong with filling your glass in the bathroom - especially if you wake up thirsty in the middle of the night and can't be bothered to go downstairs.
But the answer to the age old question apparently lies in how old your house is - and importantly, what the pipes are made from.
In more modern houses, copper and plastic are often used in plumbing, while in older houses - more commonly pre-1970 - pipes were often made using lead, which can be dangerous if absorbed into water.
In a campaign between Scottish Water and Water Safe, people were urged to check whether they have lead pipes in their homes.
It's pretty easy to find out if you have them, but if you're unsure, it's best to call a plumber. According to United Utilities, "many houses built before 1970 have lead plumbing and also a lead supply pipe (the underground pipe which connects your home to the water main in the street). If your home was built after 1970, it is unlikely to have lead pipes.
"A simple check is to look at the pipework in your kitchen. Find the pipe leading to the internal stop tap (usually under the kitchen sink). Lead pipes are dark grey (under any paint) and are soft and easily marked. If in doubt, ask a plumber to check for you."
You can find out more here.
You can also contact your local water company to test the levels and offer advice on alternatives.
Julie Spinks, director of Water Safe, said: "The plumbing in homes is the responsibility of the homeowner, so we are urging all households to spend a few minutes checking if they have lead pipes.
“To safeguard health we would recommend replacing lead pipes that supply drinking water to bathrooms and kitchens with copper or plastic ones."
And that's not all. It's generally not a good idea to drink from the hot water tap in your home. Not only can heating and cooling water multiple times lead to increased bacteria levels, but it can also cause the pipes to release harmful substances (such as lead).
We'll definitely think twice before heading to the bathroom for a drink in the night!