We claim that local e-services benchmarking studies summarized in indexes do little to enhance city managers' and academics' understanding of actual e-government performance, or to improve the e-services offered by cities. We undertook a different benchmarking approach, focused on learning best practices among cities, in late 2008 and early 2009. A benchlearning methodology (BLM) was developed, and a pilot study with 15 European cities was carried out. In this paper, we present the actual impact of the benchmarking study with respect to improvements in services, as the effectiveness of e-government benchmarking has rarely been evaluated. We discuss and analyse the results of a survey carried out in the same 15 cities four years after the pilot study. This paper presents evidence that BLM helped cities to identify good practices that they could learn from, and that some e-services were subsequently improved. The survey reveals that some changes are needed in the benchmarking methodology. The main one is the updating of the BLM bottom-up e-services catalogue, which is deeply discussed within the changing context of Smart Cities, especially the enlargement of the ecosystem of e-services to include citizens, the third sector, entrepreneurs, companies and other actors. A second one is the measurement of the adoption of e-services by citizens, also rarely assessed.