Preheat the oven to 425°. Line a sheet pan with parchment paper. In a large saucepan, bring the water, butter, salt and sugar to a boil over medium-high heat. Remove saucepan from heat. Add flour all at once. Stir until all the flour is moistened, about 1 minute. Return to heat and cook, stirring constantly, for another 1 to 2 minutes until dough forms a smooth ball.
Scrape the mixture into a mixer fitted with a paddle attachment (or use a hand mixer). Mix at medium speed. With the mixer running, add 3 eggs, 1 egg at a time. Stop mixing after each addition to scrape down the sides of the bowl. Mix until the dough is smooth and glossy and the eggs are completely incorporated. The dough should be thick, but should fall slowly and steadily from the beaters when you lift them out of the bowl. If the dough is still clinging to the beaters, add the remaining 1 egg and mix until incorporated.
Using a pastry bag fitted with a large plain tip, pipe fat lengths of dough (about the size and shape of a jumbo hot dog) onto the lined baking sheet, leaving 2 inches of space between them. You should have 8 to 10 lengths.
Use your fingers to smooth out any bumps or points of dough that remain on the surface. Bake 15 minutes, then reduce the heat to 375° and bake until puffed up and light golden brown, about 25 minutes more. Try not to open the oven door too often during the baking. Let cool on the baking sheet.
Fit a medium-size plain pastry tip over your index finger and use it to make a hole in the end of each eclair (or just use your fingertip). Using a pastry bag fitted with a medium-size plain tip, gently pipe the custard into the eclairs, using only just enough to fill the inside (don't overfill them.)