The Jevons Paradox, which got its name from the economist William Stanley Jevons, predicts that "as technology progresses, the increase in efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource". The canonical example of this is coal consumption in the early industrial era: As the machinery became more efficient at using coal, more coal was used because these efficiency gains made the whole process more economic and so it encouraged more use of the machines and more of those machines to be built.
While the Jevons Paradox is a powerful mental model that should always be kept in mind when we're talking efficiency, we should be careful not to overstate it either, because that can lead to cynicism about all efficiency progress ("Oh well, this 2x more efficient thing is pointless because somehow the Jevons Paradox will mean that we don't save any energy"). I think reality is a little bit more complex than that, and that a lot of the time the efficiency gains are worth it, even in cases where the Jevons Paradox fully applies. Let me explain what I mean by that...
First, we have to look at the specific situation. Coal usage in early factories has very different incentives than, say, gas mileage on a car or the energy efficiency of a lightbulb. With the factories, more machinery and running the machinery more equals more profit. So there's a huge incentive to go right up to the point of diminishing returns to extract as much profit as possible from the operation.
With mature technologies like residential or commercial lighting or transportation, the rebound effect tends to be small. Nascent industries, like the coal-powered steam-machines of early industrialization were very far from meeting demand for what they could produce. Any efficiency gains were immediately absorbed into increased supply to try to meet demand.
But people usually have as much light as they need in their houses and commercial properties, and people with cars drive about as many miles as they need to in a year. There's not a huge unmet demand that is being constrained by energy costs. That's why if the 70% of lightbulbs in the U.S. that are still the old inefficient models get converted to long-lasting, super-efficient LEDs, we won't see people suddenly use 4x more light. They already have enough light, relatively few people (at least in the wealthier countries) is sitting in the dark, wishing they could afford a light.
This doesn't mean that we won't end up with LED lights in all kinds of places where we wouldn't have seen lights before. A bit part of that isn't the efficiency, though, it's the fact that LEDs lights have very different characteristics when compared to fluorescents or incandescents. It's more a case of a new technology permitting new uses rather than just energy efficiency making people use more of something. So Jevons Paradox isn't the main driving factor here.
Another way to escape Jevons Paradox is to sidestep it. For example, it has often been said that hybrid and fuel efficient vehicles might encourage people to drive more, thus negating the benefit of the higher MPG. That might or might not be true - personally I think it's overstated in most cases, though I'm sure we can find individual people where that applies - but over time the point will become moot because hybrids are just a transition tech. Plug-ins are the destination, and with an electric car or plug-in hybrid that can do most of your trips on electricity, it doesn't matter much if your cost of "fuel" per mile drops significantly. By using a cheaper energy source and much more efficient electric motors, you might encourage people to drive a bit more, but the benefits of going electric are so massive (and they'll get bigger as we clean up the power grid) that it doesn't really matter. The same would be true with LED lights if you go from getting electricity from coal to solar.
Even in a theoretical scenario where you have 4x more LED lights than you had incandescents, it would still be a big improvement because of the source of the energy. But in practice, I suspect that most people will keep about the same number of lights as before and the power grid will keep getting cleaner (in good part because solar keeps getting cheaper year after year).