By Brent Stephens on July 14, 2013
A few nights ago, we were having a discussion among the research group about some of the advantages and disadvantages of the USGBC LEED rating system, and what everyone really thought about the system. This provoked me to put my own thoughts and opinions in writing. So here is my take….
The US Green Building Council (USGBC) was founded in 1993 as a membership-based nonprofit organization that “promotes sustainability in how buildings are designed, built, and operated.” In 1998, the USGBC released its pilot version of the Leadership in Energy and Environmental Design (LEED) rating system for new construction (NC). They now have nine separate LEED rating systems, from LEED-NC to LEED for Homes, Schools, Healthcare, Existing Buildings, Neighborhood Development, and a few others.
In LEED-NC 2009, the widely used commercial building standard, there are 100 points available from a checklist of items across five categories: Sustainable Sites, Water Efficiency, Energy and Atmosphere, Materials and Resources, and Indoor Environmental Quality, plus a few extra points for “innovation in design” and “regional priorities.” To obtain a LEED-certified building, you have to achieve 40-49 points; 50-59 points will get you Silver; 60-79 Gold; and 80+ Platinum.
Generally, the following topics are covered in each category (with maximum number of points from that category in parentheses):
1. Sustainable Sites (26 possible points). Increased urban density, brownfield redevelopment, locations near public transit, facilities for alternative vehicles, stormwater management, habitat preservation/restoration, heat island effect mitigation, and light pollution mitigation.
2. Water Efficiency (10 possible points). Reduced water use in the building, water efficient landscaping, and innovation in wastewater treatment onsite.
3. Energy and Atmosphere (35 possible points). Minimum energy performance (meeting ASHRAE 90.1), additional points for reduced energy consumption, on-site renewables, enhanced commissioning, measurement and verification of energy use, control of refrigerant releases, and purchasing grid-connected green energy.
4. Materials and Resources (14 possible points). Provide recycling bins, reuse of materials for walls, floors, roofs, and others, managing construction waste, sourcing materials locally, using renewable materials, and using materials with recycled content in them.
5. Indoor Environmental Quality (15 possible points). Prohibit indoor smoking, meet ASHRAE 62.1 ventilation rate minimums, increase ventilation, minimize effects of material emissions during construction and right after construction, use of low-VOC content materials in sealants, adhesives, paints, coatings, and flooring, proper use of chemical storage facilities, lighting and thermal comfort controls and verification, and daylighting.
Now, the question remains: If you build a building whereby you succeed in achieving 40 to 80 of these points, is the result a “sustainable building?”
I think that’s a bit of a loaded question, but it’s worth looking into some recent research that has tried to address this question, primarily in terms of energy performance.
2008 NBI study
One of the first studies to attempt to address the question “Do LEED-NC building save energy?” was published by the New Buildings Institute (NBI) in 2008. In the report, which was written by NBI for USGBC, the authors used actual measured energy performance from 121 LEED-NC certified buildings that had been built from 2000 to 2006. This sample represented 22% of the 552 LEED-NC certified buildings that had been built at the time under that version of LEED. They normalized annual energy use by the square footage of each building, reporting the EUI (energy use intensity in kBTU per square foot per year) for each building. They concluded that, on average, LEED-NC buildings were indeed saving energy compared to the national building stock. The median EUI for LEED-NC buildings in the sample was 24% lower than — and here’s an issue with their study — the national average for all existing commercial buildings in the US building stock. Interestingly, LEED Silver buildings had an average EUI about 7% lower than LEED-certified alone, and LEED Gold or Platinum (lumped together) had an average EUI about 24% lower than LEED-certified alone. They also threw out 21 “high energy type” buildings from the sample because they were not used for typical commercial purposes; they were other building types with more intense indoor energy using activities.
In the NBI report, they also took a look at how buildings were performing relative to what they were predicted to perform. They showed decent correlation between measured EUI and design EUI, although about half of the buildings used less than the predicted EUI and about half used more!
Another important conclusion from this study was that although the majority of LEED certified buildings used less energy than non-LEED buildings, according to their analysis, several (about 15 of them) actually used more energy than a baseline code building. Finally, they suggested LEED could use a few improvements, including quality control, follow-up measurements, advanced commissioning, and comparisons to modeled performance, all of which I agree with.
Lies, Damn Lies…
Just after the NBI study was released, Henry Gifford, an influential mechanical engineer and building scientist in NYC posted a critique of the study on his website, energysavingscience.com. In his report, Gifford re-examined the NBI study data and came up with an entirely different conclusion from the NBI report. For one, Gifford had major issues with the representativeness of the sample buildings, suggesting the people that responded to the survey may be more likely to have achieved energy savings in their buildings than ones who declined to participate in the survey.
An even bigger issue raised by Gifford was with the NBI comparison of LEED buildings to existing commercial buildings. He (correctly IMO) argues that is not a fair comparison because all LEED buildings were built after 2000, so some inherent level of improvements in energy efficiency for all buildings built since 2000 are not taken into account if a new LEED building is compared to a commercial building built in, say, 1960. So the better approach is to compare LEED buildings to non-LEED buildings that were built in the same time frame (i.e., the early 2000s). According to the US Commercial Building Energy Consumption Survey (CBECS), the mean EUI for these buildings is 81.6 kBTU per square foot per year compared to 91 kBTU/sq ft for all buildings. So the baseline in the NBI study was wrong to begin with!
Additionally, the comparison of LEED medians to CBECS means is not a valid approach. Looking back at the NBI data, it was found that the actual mean EUI of LEED buildings was actually 29% higher than the 2000s-built CBECS buildings! Gifford goes on to conclude in his analysis:
“The LEED system has changed the market for environmentally friendly buildings in the US, but there is an enormous problem: the best data available shows that on average, they use more energy than comparable buildings. What has been created is the image of energy efficient buildings, but not actual energy efficiency.”
Gifford even got written up in the NY Times for his work.
2009 Newsham et al. study
Issues with the NBI report continued to be raised in the literature. In August 2009, researchers at the National Research Council Canada published an article in Energy and Buildings entitled “Do LEED-certified buildings save energy? Yes but…” In this paper, the authors conducted a re-analysis of the same data supplied by the NBI study on the same 100 LEED-certified commercial and institutional buildings (excluding the 21 high-energy use buildings that NBI also excluded). They also compared EUI to the general US commercial building stock and found that on average LEED buildings used 18-39% less energy per floor area than their “conventional counterparts.” However, 28-35% of LEED buildings were found to use more energy per floor area than the same “conventional counterparts,” suggesting that while on average LEED may be saving energy, it is by no means a guarantee for energy savings.
The NRC study also pointed out that the original NBI study suffered from some other analytical weaknesses. For one, median EUIs were compared to mean EUIs across the existing commercial building stock, which is not exactly statistically sufficient, similar to Gifford’s issues. Additionally, no statistical tests were performed on their data in the NBI report. To increase the specificity of the comparisons, the NRC study surveyed the US Commercial Building Energy Consumption Survey (CBECS) to find “conventional counterparts” for each of the 100 LEED buildings used in the NBI study. The idea here was to attempt to normalize the analysis not only for building type (e.g., compare commercial office versus commercial office and K-12 educational versus K-12 educational buildings), but also for climate, which has a huge impact on space conditioning energy used regardless of building type, and for year of construction and overall floor area. Therefore, t-tests could be performed on the matched LEED-certified buildings and their “conventional counterparts” of similar type in similar climates. From there, they arrived at the conclusion that LEED buildings still, on average, used less energy per floor area than their conventional existing partner buildings, but that one-quarter to one-third actually used more.
The NRC study also explored energy performance in relation to the number of Energy and Atmosphere credits that each building received in their LEED rating. There was a trend in that for every additional energy performance credit achieved, EUI was ~3.4 kBTU per square foot per year lower (or about 4% lower per Energy Performance point achieved), albeit with a lot of scatter. IMO, this is a somewhat encouraging finding. Not all LEED credits have to do with energy performance, but if you aim for better energy performance by following the standard, it appears you can actually achieve it.
2009 Scofield study
Later in 2009, John Scofield of Oberlin College published a follow-up article in Energy and Buildings, “Do LEED-certified buildings save energy? Not really…” In this article, Scofield took issue with the Newsham et al. re-analysis of the NBI data. One primary difference in Scofield’s analysis is that he took into account “source energy,” which accounts for both on-site energy used (which is measured in the NBI study) and off-site losses associated with generation and distribution of electricity. (The story here is that if you burn natural gas for heating and other uses on-site, your conversion efficiency from fuel to useful energy is typically 80-95+%; if you burn fuel to generate electricity, then use that electricity on-site, your conversion efficiency is typically closer to 30-40%, depending on fuel type, distribution lines, and power plant efficiencies).
Scofield also re-analyzed the data where instead of using mean EUI for a collection of buildings where all buildings are treated equally regardless of how big they are and how much energy they used, he weighted the buildings according to size. He did this by simply summing the total amount of energy used by all buildings in sample divided by the total amount of floor area of all buildings, which gives a true mean of EUI across the sample.
He considered this averaging method more representative of the average EUI of these buildings as a whole, and in doing so he found that yes, indeed, LEED buildings saved some site energy on average compared to conventional counterparts, but only 10-17%, not the 18-39% suggested by Newsham et al. When he combined this method with source energy analysis, accounting for electricity losses, he found no statistically significant differences in the LEED-certified or conventional CBECS counterpart building samples!
In this study, Scofield concluded the following :
“I found that LEED offices used, on average, 17% lower site energy than CBECS offices, but no less source energy. The reason, of course, is that large buildings dominate the energy consumption of a set of commercial buildings. Despite the fact that many of the smaller LEED buildings outperform their conventional counterparts, it matters little because they do not contribute to the total energy nearly as much as a handful of larger buildings. LEED proponents may think it unfair that a few large buildings should dominate the total energy—but that is physics.”
2012 Oates and Sullivan study
A few years later in 2012, researchers at Arizona State University published an article in the ASCE Journal of Construction Engineering and Management, “Postoccupancy energy consumption survey of Arizona’s LEED New Construction program.” In this work, Oates and Sullivan reported on energy use of 25 LEED-certified in Arizona, which represents about half the total number of LEED buildings built in Arizona at the time. Their results suggest that on average Arizona’s LEED-NC buildings of medium-use type performed better than the national average, but actually performed worse than buildings located in similar climates. The LEED buildings classified as “high energy” use types performed no better than conventional “high energy” use types. The authors concluded:
“The study’s results demonstrate to industry practitioners and researchers that the LEED NC rating system’s energy strategies fail to meet modeled efficiencies, thereby highlighting a need for ongoing scrutiny and diligence when measuring sustainability and efficiency within Arizona’s built environment.”
So what does this all mean?
These studies represent some of the largest samples of LEED vs. non-LEED energy performance comparisons. I wish we had more (and there may be more out there that I’m not aware of). But from what I can tell from these studies, I conclude the following:
- Many LEED buildings have better energy performance than conventional counterparts, but many LEED buildings perform worse.
- LEED-certification does not mean a building has increased energy performance, by default.
- Assuming that LEED buildings are more energy efficient by default is a dangerous assumption that undermines the entire basis of LEED.
- However, to play devil’s advocate, energy performance accounts for only 35% or less of the total possible LEED points. You only need 40 points to get certified, and many of the other available points can be achieved more readily and inexpensively than some of the energy performance points, so it makes sense to me that energy performance is not better by default.
- Seeking more energy performance points appears, however, to lead to greater energy performance, which is logical and generally acceptable.
- The LEED point system appears entirely arbitrary to me. I have never seen an analysis of the true impacts of each point in the system on any helpful metric or outcome, whether energy use, cost, greenhouse gas emissions, or other toxicity and life cycle impacts. I think it is time that we evaluate what the true point system should be — and if that should be any different than it is now — using a range of case studies. The point system should be reverse engineered from measurable outcomes related to sustainability (although we could argue all day about what that actually means).
- Many of the individual points available to pursue are not well thought out. For example, you can achieve points by using paints and adhesives with low-TVOC (total volatile organic compound) content. While that makes sense at a high level, recent research has shown that actual emissions of VOCs from paint are quite complex and don’t necessarily relate to TVOC content in the paint itself. Additionally, controlling TVOCs may not be as important as controlling certain individual VOCs. There are many more examples like this where the LEED point system does not appear necessarily informed by recent research.
- At the end of the day, LEED has done wonders for increasing awareness of green buildings and sustainability, which should not be overlooked no matter how imperfect the system is and continues to be. However, the building industry MUST get away from assuming LEED = energy efficient; this will be harder said than done.
- LEED must keep moving forward with better commissioning and performance requirements, which the new version due out in the fall is working towards.
