Open Forum #81 What Gets Measured Gets Addressed How the US Measures Internet Use

Summary

This discussion focused on measuring internet use in the United States, featuring presentations from the National Telecommunications and Information Administration (NTIA) and the U.S. Census Bureau. The session began with a historical video from the 1990s showing early efforts to address the “digital divide” between Americans with and without access to information technology. NTIA officials presented their Internet Use Survey, which has been the longest-running federal data collection on computer and internet use since 1994, conducted as a supplement to the Current Population Survey with tens of thousands of households interviewed monthly.

The Census Bureau representatives discussed the American Community Survey (ACS), which measures computer and internet access among 3.5 million households annually and provides geographically granular data down to neighborhood levels. Both agencies acknowledged limitations in their current data collection methods, particularly the challenge of obtaining reliable estimates for smaller geographic areas and keeping pace with rapidly changing technology. To address these gaps, NTIA and the Census Bureau introduced Project LEIA (Local Estimates of Internet Adoption), which uses advanced statistical modeling techniques called small area estimation to produce more detailed local internet adoption data.

Project LEIA combines direct survey data with auxiliary information like income levels and broadband infrastructure availability to generate reliable single-year estimates for all U.S. counties, rather than just the quarter of counties with sufficient population for traditional survey methods. During the interactive discussion, international participants raised questions about sustainability measurements, data quality assessment, capacity building opportunities, and how collected data influences policy decisions. The session concluded with participants sharing experiences from different countries and expressing interest in continued collaboration on internet measurement methodologies.

Keypoints

## Major Discussion Points:

– **Historical Evolution of Digital Divide Measurement**: The discussion began with historical context from the 1990s, showing how the “digital divide” between technology “haves and have-nots” became recognized as a critical economic and civil rights issue, leading to the establishment of long-running federal data collection efforts.

– **Two Primary Survey Methods for Internet Use Data**: Presenters detailed NTIA’s Internet Use Survey (running since 1994, supplementing the Current Population Survey) and the Census Bureau’s American Community Survey questions (added in 2013), explaining how these capture different aspects of internet adoption, device usage, and online activities.

– **Project LEIA – Advanced Local Estimation Techniques**: A significant focus on the new “Local Estimates of Internet Adoption” initiative using small area estimation and machine learning to provide more granular, single-year county-level internet adoption data, addressing gaps in geographic coverage of traditional surveys.

– **Data Limitations and Methodological Challenges**: Discussion of survey limitations including declining response rates, difficulty measuring rapidly changing technologies, inability to capture internet quality/infrastructure readiness, and challenges in providing estimates for smaller geographic areas.

– **International Collaboration and Capacity Building**: Questions from international participants about training opportunities, data sharing, and how measurement data translates into targeted policy interventions for digitally excluded populations.

## Overall Purpose:

The discussion aimed to present current U.S. federal approaches to measuring internet use and adoption, introduce new statistical modeling techniques for more granular local estimates, and foster international dialogue about best practices, methodological improvements, and collaborative opportunities in internet measurement.

## Overall Tone:

The discussion maintained a consistently professional, collaborative, and educational tone throughout. It began as a formal presentation but became increasingly interactive and engaging during the Q&A portion. The tone was optimistic about technological advances in data collection methods while acknowledging ongoing challenges. International participants brought a global perspective that enriched the conversation, and the overall atmosphere was one of knowledge-sharing and mutual learning rather than debate or criticism.

Speakers

**Speakers from the provided list:**

– **Video**: Historical testimony from the Honorable Larry Irving presenting findings from “Falling Through the Net, Defining the Digital Divide” study released by the Commerce Department in 1999

– **Jaisha Wray**: Associate Administrator for the Office of International Affairs and Acting Associate Administrator of the Office of Policy Analysis and Development at the National Telecommunications and Information Administration (NTIA)

– **Susan Chalmers**: Internet Governance Team Lead in the Office of International Affairs at NTIA

– **Luis Zambrano Ramos**: Senior Policy Advisor in NTIA’s Office of Policy Analysis and Development

– **Andrew Flavin**: Leads the Digital Policy Team in NTIA’s Office of International Affairs

– **Rafi Goldberg**: NTIA policy expert with 14 years of experience at the agency

– **Leslie Davis**: Census Bureau’s Subject Matter Expert on Computer and Internet Use as measured by their surveys

– **Heather Keene**: Works in the same division as Leslie at the Census Bureau, brings modeling expertise to internet and computer use measurement

– **Audience**: Multiple audience members asking questions during the open forum discussion

**Additional speakers:**

– **Michael Lewis**: Mentioned as someone who would discuss the NTIA Internet Use Survey and its history, but did not appear to speak in the transcript

– **Nenad Dorlich**: Audience member who asked questions about sustainability data collection and comparison with commercial service providers

– **Robert**: Online participant from Uganda working for Youth in Technology and Development Uganda, asked about capacity building training

– **Joshua**: Audience member from Uganda ISOC chapter who asked about disaggregation of internet measurement in schools and institutions

# Measuring Internet Use in the United States: Federal Data Collection Approaches and International Collaboration

## Executive Summary

This forum examined federal approaches to measuring internet use and adoption in the United States, featuring presentations from the National Telecommunications and Information Administration (NTIA) and the U.S. Census Bureau. The session began with historical context from the 1990s digital divide research, followed by detailed presentations on current survey methodologies and the innovative Project LEIA small area estimation initiative. The discussion included questions from international participants about capacity building, data comparability, and policy implementation.

## Historical Context: The Digital Divide Foundation

The session opened with a video clip from the 1990s featuring Larry Irving discussing the Commerce Department’s “Falling Through the Net, Defining the Digital Divide” study. This historical footage provided context for understanding how the digital divide between technology “haves and have-nots” was first identified as a critical policy issue affecting American access to technology.

## NTIA Internet Use Survey

Luis Zambrano Ramos, Senior Policy Advisor in NTIA’s Office of Policy Analysis and Development, presented the NTIA Internet Use Survey, which represents the longest-running federal data collection effort on computer and internet use, operating continuously since 1994. The survey operates as a supplement to the Current Population Survey (CPS), interviewing tens of thousands of households monthly.

The survey captures comprehensive data on device usage, online activities, and internet access technologies, measuring not merely access but the sophistication and variety of internet usage patterns across demographic groups. The survey methodology undergoes continuous refinement through a public comment process that ensures stakeholder input in data collection design.

Zambrano Ramos highlighted the public availability of raw data through GitHub repositories and explorer tools designed for researchers and policymakers, demonstrating the agency’s commitment to data transparency and accessibility.

## American Community Survey (ACS)

Leslie Davis, the Census Bureau’s Subject Matter Expert on Computer and Internet Use, explained how computer and internet questions were added to the ACS in 2013 following Congressional directive through the Broadband Data Improvement Act. The ACS provides nationally representative data on internet adoption and computer ownership for 3.5 million households annually.

The ACS underwent significant methodology improvements in 2016 to address question clarity and response reliability issues. Davis emphasized that the ACS enables analysis of broadband adoption disparities at sub-state geographic levels, though with important limitations: one-year data is limited to counties and places with populations exceeding 65,000, while five-year data can reach census tract level but sacrifices timeliness for geographic coverage.

Davis detailed the interactive mapping tools and pre-tabulated products available through data.census.gov, making statistical information accessible to diverse user communities.

## Project LEIA: Local Estimates of Internet Adoption

### Technical Approach

Rafi Goldberg, NTIA policy expert, and Heather Keene, Census Bureau modelling expert, presented Project LEIA as an innovative solution addressing geographic limitations in traditional survey methods. Announced last fall, the project employs small area estimation techniques that combine direct survey data with auxiliary predictors such as income levels and broadband infrastructure availability.

The statistical approach blends direct ACS estimates with indirect regression estimates, weighted by relative precision for each county. This methodology enables production of reliable single-year estimates for all U.S. counties, rather than just the quarter of counties with sufficient population for traditional survey methods.

### Results and Applications

Project LEIA produces the first-ever experimental single-year estimates of household internet adoption for every U.S. county. This advancement in geographic granularity enables state broadband offices to utilize survey data for designing targeted programs. Goldberg provided specific examples, noting that while Los Angeles County has sufficient sample size for direct estimates, smaller counties in Texas do not, making the modeled estimates particularly valuable.

### Future Development

The research team outlined plans for expanding Project LEIA’s capabilities, including refining the county-level model to produce official data products, exploring census tract-level estimates, and applying machine learning techniques to bring detailed internet use data from NTIA surveys to ACS datasets. Keene mentioned plans to incorporate additional predictors including urban/rural measures and STEM occupation concentrations.

## International Engagement and Questions

The session included participation from international attendees, with Jaisha Wray, Associate Administrator for NTIA’s Office of International Affairs, and Susan Chalmers, Internet Governance Team Lead, facilitating discussion.

### Key Questions Raised

**Capacity Building**: Robert from Youth in Technology and Development Uganda requested training on internet measurement methodologies. Speakers suggested working through existing institutions like the U.S. Telecommunications Training Institute.

**Environmental Sustainability**: Nenad Dorlich asked whether agencies collect data on electricity consumption or CO2 emissions related to internet usage. Speakers acknowledged this data is not currently collected.

**Data Disaggregation**: Joshua from Uganda ISOC chapter asked about measuring internet access in schools and institutions beyond household-level data. Speakers noted that while other agencies collect some institutional data, comprehensive integration remains limited.

**Infrastructure Quality**: A RIPE NCC representative questioned measuring internet infrastructure readiness for next-generation applications like AI and AR, highlighting that access doesn’t necessarily indicate capability for advanced applications.

**Policy Implementation**: An international think tank representative asked how data collection translates into targeted initiatives for digitally excluded populations, emphasizing the gap between measurement and effective intervention.

## Technical Challenges

Speakers acknowledged several ongoing challenges:

– Declining survey response rates affecting data reliability

– Sample size constraints limiting geographic granularity

– The need to balance technological currency with time series comparability

– Integration challenges across multiple data collection efforts

## Data Accessibility and Tools

Both agencies emphasized their commitment to open data access. NTIA provides raw data through GitHub repositories and specialized explorer tools, while the Census Bureau offers interactive mapping tools and pre-tabulated products through data.census.gov. This approach enables independent analysis and supports evidence-based policy development across government levels.

## Conclusion

The forum demonstrated significant progress in federal internet measurement capabilities, from the foundational digital divide research of the 1990s to today’s sophisticated statistical modeling approaches like Project LEIA. While current methodologies provide valuable insights into internet adoption patterns, the discussion revealed ongoing challenges in areas such as environmental impact measurement, infrastructure quality assessment, and translating data into effective policy interventions.

The international interest in U.S. methodologies suggests opportunities for knowledge sharing and capacity building, while the technical innovations presented indicate continued evolution in measurement approaches to address the changing digital landscape.

Video: and so at this time we’d like to ask the Honorable Larry Irving to present testimony. Thank you. Thank you, Mr. Chairman. I’d like to thank you and the members of the subcommittee for this opportunity to testify today on the findings of Falling Through the Net, Defining the Digital Divide, the study released by the Commerce Department earlier this month. President Clinton and Secretary Daley released Falling Through the Net on July 8th, 1999 in Los Angeles during the President’s New Markets Tour. And during the tour, the President and Secretary Daley discussed the fact that even though information technology underlies much of our nation’s economic growth, far too many Americans are left out of the digital economy. And as a result, the digital divide, that is the divide between the haves and have-nots in information technology, has become a critical economics and civil rights issue. Access to new technologies such as the computer and the Internet will be keys to the future economic success of any American business, community, or individual. And increasingly, Americans use the net to find jobs, contact colleagues, locate public information, or take courses online. Electronic commerce is helping small companies compete and entrepreneurs in rural, remote, and traditionally underserved areas reach out to the rest of the world. Familiarity with new technologies will also prepare more Americans for the high-tech workplace of the 21st century. Because of the increasingly important role of these new technologies, Secretary Daley concluded that ensuring access to the fundamental tools of digital economy is one of the most significant investments our nation can make. And as we enter the 21st century, it will become even more essential to ensure that all Americans, rich or poor, urban or rural, black or white, Hispanic or Native American, can reap the benefits of these new technologies. Falling through the net provides a starting point in bridging the gap between the nation’s information rich and poor. This is our third report examining census data, looking at the digital divide. And we anticipate that it will serve as an important diagnostic tool to assist policymakers in the private sector in formulating methods to provide greater access for more Americans. And today I’d like to provide for the subcommittee several slides showing some of our key findings. These slides illustrate that overall Americans are far more connected than they have been in years past. On the other hand, we have also found that there are alarming disparities based chiefly on income, education, race, and geographic location in which group of Americans have computers, and who is online. Hmm, it’s gonna be interesting as I read my testimony. Equally disturbing, many of these disparities are growing. And let me turn to the first slide. This shows… And everyone will be in the dark if we don’t get online.

Jaisha Wray: Great, thank you all for joining us today. My name is Jaycia Ray and I am the Associate Administrator for the Office of International Affairs and the Acting Associate Administrator of the Office of Policy Analysis and Development in the National Telecommunications and Information Administration, or NTIA, in the United States. What you just saw is a video from NTIA in the 1990s discussing the early days of measuring Internet use in the United States. As you will hear, this issue continues to be of high importance and the work continues on to this day as well. Today we have a distinguished set of speakers from NTIA and the U.S. Census Bureau who will discuss the past, present, and future of measuring Internet use in the United States. Located within the U.S. Department of Commerce, NTIA is the executive branch agency principally responsible by law for advising the President on telecommunications and information policy issues. NTIA’s programs and policymaking focus largely on expanding broadband Internet access and adoption in America, expanding the use of spectrum by all users, and ensuring that the Internet remains an engine for innovation and economic growth. Also located within the Department of Commerce, the United States Census Bureau is the nation’s largest federal statistical agency with the mission to serve as the leading provider of quality data about the nation’s people and economy. The Census Bureau provides high quality, timely, and relevant data products that determine how legislative seats are distributed across the nation. It also supports evidence building in government and policymaking, planning decisions about community services, and the annual distribution of federal funds to local, state, and tribal governments. Together, subject matter experts from these agencies will share details about how they measure Internet use, highlighting promising new initiatives and issues that require further examination in this ongoing effort. Following their overview, we want to open the floor for robust interactive discussion with people both in the room and online. We want to hear how other countries are measuring Internet use and what we can learn from those experiences. In particular, what are the missing pieces in measuring Internet use, and how can the international community collaborate in this space? We hope that this open forum will serve as an opportunity to bring experts from around the world together to advance the state of knowledge for one of the most important aspects of digital connectivity, understanding how people use the Internet. So to kick things off, I will ask my colleagues to briefly introduce themselves.

Susan Chalmers: Good afternoon. My name is Susan Chalmers, and I am the Internet Governance Team Lead in the Office of International Affairs at NTIA.

Luis Zambrano Ramos: And I’m Luis Zambrano Ramos. I’m a Senior Policy Advisor in NTIA’s Office of Policy Analysis and Development.

Andrew Flavin: Hi, I’m Andrew Flavin. I lead the Digital Policy Team in NTIA’s

Luis Zambrano Ramos: Office of International Affairs. Rafi, are you online?

Rafi Goldberg: Good afternoon. My name is Rafi Goldberg, and I’ve been working on policy at NTIA for the past 14 years. And I will pass it on to our Census Bureau friends.

Leslie Davis: Great. Thank you, Rafi. My name is Leslie Davis, and I’m the Census Bureau’s Subject Matter Expert on Computer and Internet Use as measured by our surveys.

Heather Keene: Hi, good afternoon. I’m Heather Keene. It’s morning for us. I work in the same division with Leslie, and you’ll meet Mike after me. They’re the Subject Matter Experts on Internet and Computer Use, and I bring modeling expertise to the table. I’m going to pass it to you, Mike.

Jaisha Wray: All right. So in terms of our run of show today, first we will hand it over to Michael Lewis, who will discuss the NTIA Internet Use Survey and its history. Following that, Leslie will talk about the U.S. Census Bureau’s decades-long work in the American Community Survey. After Leslie, Rafi will highlight the latest project to produce local estimates of Internet adoption. Heather will then dive deeper into the technical details of this project. And following the presentation, Susan and our other Subject Matter Experts will help steward the open forum discussion. So with no further ado, we’ll get started, and over to Luis.

Luis Zambrano Ramos: Great. So thank you so much, Jaisha Wray. So I am going to focus on one of NTIA’s most important contributions to the field, the NTIA Internet Use Survey. This survey is a supplement to the current population survey, which is a survey that’s sponsored jointly by the U.S. Census Bureau and the U.S. Bureau of Labor Statistics. It is a monthly survey, which is primarily known as the source of some of our official labor statistics. Agencies such as NTIA have the opportunity to sponsor supplemental questions during particular months. Tens of thousands of households are interviewed for the CPS every month, providing a large, nationally representative sample of American households. We have worked for over 30 years with our partners at the Census Bureau on this survey. In fact, the Internet Use Survey is the longest-running federal data collection on computer and Internet use in the United States. We have sponsored a grand total of 17 surveys in the field since 1994, and we’re planning to get our next survey out in November of this year. So here you can see a timeline of our collections. Over time, we have transitioned from print publications of our findings to a more web-centered approach to present the results. We have blog posts and other analyses of the data, and we also have a nifty, publicly available explorer tool that I will showcase in a bit. One very important component of the NTIA Internet Use Survey is the ability to obtain input from the public on the survey instrument itself. We include a draft version of the survey instrument and seek comment from the public before we issue the survey. This forum actually is particularly timely, given that last week, on June 17th, we published a notice and request for comment for our upcoming survey later this year in 2025. So with all this said, what can we learn from the survey? So in addition to learning just about Internet use generally, we can also learn things like devices and Internet access technologies, details about how people use the Internet, including locations and certain online activities. We also have other questions, such as questions about how people use the Internet. This is a slide that you can get using our explorer tool, which I will show in a bit. This slide breaks down device usage, perhaps not surprisingly for example, we have noticed that smartphone use has gone up since 2011, while desktop computer use has decreased. And here are a few online activities that we have measured and how they have trended over time, things like social media use, telework, and the like. So as I mentioned, we have a publicly available explorer tool that we call the NTIA Dead Explorer that allows us to play with this data. For example, if I am searching for device use, specifically laptop use, you can look at that information throughout the years. You can look at the total number in millions and percentages over the years. You can also get a table that gives you a result for each of the U.S. states and breaks them down along with numbers, percentages, and also things like confidence intervals. So again, here is just how the tool is being applied, and I’m sorry, I think we’re having some technical difficulties, but moving on to sort of the availability of this data, if you want to dive deeper, we make all the underlying raw data publicly available. We actually have a GitHub repository with sample code, and at the end of the presentation we’ll actually give a link to our website. And there you can see some of the publicly available data sets that we have. And before I pass it on to my Census colleagues, last year we actually celebrated the 30th anniversary of the Internet Use Survey, and we threw a little party. You just saw that slide, and we were able to put it on a cake, and unfortunately there’s no cake today, but I welcome you all to join in the celebration. And with that, I will pass it on to my colleague Leslie from Census.

Leslie Davis: Great. Thank you so much, Luis. In addition to the NTIA Internet Use Survey, the Census Bureau measures computer and Internet access on the American Community Survey, or the ACS, which is the Census’s flagship demographic survey. This is a nationally representative survey that serves as the premier source for timely, detailed population and housing information for the U.S., and it provides local and national leaders with information needed for programs, economic development, emergency management, and more. It is fielded annually to a sample of 3.5 million households nationwide, and also additional group quarters, and it asks questions on over 40 topics, including Internet access. Next slide, please. So in 2008, U.S. Congress directed federal agencies through the Broadband Data Improvement Act to improve their data collection on broadband use and subscription, and this act specifically called upon the Census Bureau to add questions to the ACS that measured, A, whether persons at households use or own computers at their address, and, B, whether persons at that address subscribe to dial-up or broadband Internet service at their address. The computer and Internet questions were added to the ACS in 2013. Next slide, please. So the initial set of questions that were asked on the ACS are shown on the left-hand side of this screen, and they were designed to capture computer ownership, Internet adoption, as we measure by a connection to the Internet, and Internet subscription type. As a result of us measuring adoption, the ACS data does not offer insights on the level of Internet use within a household or the extent of digital literacy skills of a particular respondent. Furthermore, if a respondent says that they don’t have a subscription, the ACS data does not tell us if there isn’t an Internet service provider thus not allowing for the subscription or if it’s due to another reason, such as the cost of the subscription. The ACS questions were revised in 2016, which you can see on the right-hand side of the screen, and their current versions remain as they were in 2016. These changes that we made were implemented after feedback received from survey methods experts and undergoing thorough content testing, and the changes were aimed to address potential confusion with question wording, response categories, and definitional clarity. The first question you’ll see on the screen is our changes to device ownership question, and given the focus of today’s talk, I’m going to switch to the next question, which asks about Internet access. Here we replaced the language regarding subscriptions to paying a provider or cell phone company, which helped to reframe what it means to subscribe or adopt to the Internet for our respondents. Next slide, please. As for our Internet types question, we altered the language of mobile broadband plan to cellular data plan for clarity, and this change alone improved our response reliability drastically. We also collapsed the high-speed categories of cable, fiber, and DSL to improve question clarity and also distinguish this category, which is fixed to a particular location, and compare it to the mobile broadband option. Again, here we removed language related to subscription and changed it to access to the Internet, which improved our response rates. As you can see on this question and the device type question before, we also offer the opportunity for respondents to write in a particular Internet service type or device type that they may not see on the questionnaire. Clerical coders at our headquarters can match these write-ins to our categories as best as possible and reassign those write-ins as needed. So, for example, if a respondent wrote in Comcast for their Internet service type, which is a provider of cable and fiber Internet in the United States, we can typically assign that respondent to high-speed broadband on the back end. Next slide, please. So, based on the ACS, the Census Bureau publishes one-year and five-year estimates, and the main difference between the two is the level of geographic granularity captured in the estimates. Our one-year data can go down to the county and place levels for the geographies that match our population threshold of 65,000 and over, which helps us ensure confidentiality of our respondents and ensure reliability. The five-year data, which brings together 60 months of data, can go down even smaller to the census tract and block group level, as you can see on the hierarchy on the screen. Next slide. The tables on the screen provide some context for how small these geographies are. So, for example, for the one-year data, our counties are the largest sub-state geography. They are highly variable in geographic area and population size. So, for example, Los Angeles County in California has about 10 million residents, whereas Armstrong County in Texas has about 2,000. Counties in the U.S. have some government oversight in many states. Places, on the other hand, represent a town or a settled concentration of people within a state and can often be identified by a name. They sometimes have governmental functions, and their median size is about 1,000 people. Our five-year data can go down to the tract level, which is generally represented to be neighborhoods. They have an optimum size of about 4,000 people, and even smaller than that is the block group. The block group is a group of comprised of blocks, which are statistical areas bounded by visible and non-visible boundaries, and they’re comprised of about 600 to 3,000 people. As I mentioned, the one-year data is less geographically granular, which helps us ensure the reliability of those estimates and our confidentiality of survey respondents. So, as a result, we were able to publish. 27% of counties and 2% of incorporated places in our nation for the one-year data set in 2023. Our five-year data in contrast has a greater sample size and better reliability for smaller geographies and as a result we were able to publish all estimates. Next slide please. Using the ACS computer and internet data that we collect, the Bureau publishes 14 tables on device ownership and internet adoption by select demographics and geographic breakdowns. The data is available on our data tool which is data.census.gov and data users on that tool can access pre-tabulated products and use tables to create custom maps as I did to create the map on the screen here which shows the percent of people age 65 and older with a broadband subscription and computer by state. In addition to the tables and maps, data users can use our public use microdata sample or our PUMS data to create custom tables. This PUMS data is a sample, a subsample of the ACS microdata and it goes through many disclosure processes to protect confidentiality. Next slide please. In addition to our pre-tabulated products, we also produce written reports based on the ACS data. This most recent report investigated computer and internet use trends between 2013 and 2021. It also explored disparities in broadband adoption at the county level and included a great section on the demographics of smartphone-only households in the United States. Next slide. We also author shorter blog-style publications called America Counts Stories and here I have two examples, one pertaining to tablet ownership among households with children and the other exploring broadband adoption disparities among tribal households. Next slide. So what are the strengths of the ACS data? As I mentioned, it has a large sample size which allows us to publish geographically granular data. So for example, if you were interested in studying broadband adoption at the sub-neighborhood level, our five-year data from the ACS could help you get there. Additionally, the ACS offers detailed demographic data available for sub-state geographies which helps academics, policy makers, community stakeholders, and more explore internet adoption disparities by different factors like income, education, and rurality. We also publish the data on a frequent basis on an annual basis and it undergoes rigorous verification and reliability checks prior to us publishing it and it also provides a helpful time series of data as it dates back to 2013.

Rafi Goldberg: Great. Thank you, Leslie. Hopefully, my first slide is up although I cannot see it at the moment. But the, you know, the NTIA Internet Use Survey and the ACS, thank you, have served as the basis for a ton of important research and analysis for many years now and they continue to be vital data sources for understanding internet use in the United States. But we’re also very much aware that these surveys alone are unable to answer every relevant question and recently we’ve been spending a lot of time thinking about how to address some of the gaps in our data and to ensure we’re fully equipped in the future to understand our progress and challenges. Next slide, please. We learn a ton from household surveys but they’re not without their limits. For example, it’s pretty common for a colleague or a member of the public to ask me what internet use is like in their particular area and unfortunately, our current ability to answer that question is limited. Even with a huge sample size of our surveys, those numbers start to get small very fast if you try to zoom in too far on the map. For less populous areas, as Leslie mentioned, we need to aggregate five consecutive years’ worth of ACS data. Now, you know, even if you can work with the three basic internet use questions that are on the ACS, five-year estimates, while, you know, very useful in a lot of cases, are not ideal when you’re trying to understand the impacts of relatively fast changes like the introduction of a new program. I’m excited to say that NCIA and the Census Bureau are working to fill this gap, which will be the focus of the rest of our presentation, but just want to note that we also have some other challenges in this field. For example, while our surveys have fairly high response rates, they have been falling over time, and there’s a lot of work going on to try to address that. Internet use is also very much a field where the technologies and even the words that we use to describe those technologies are changing all the time, so it’s always a balancing act between keeping up with those changes in our surveys and being careful not to break good time series comparisons. Finally, while we have data on online activities from the NCIA Internet Use Survey, we are limited in our ability to measure important workforce and other skills. Next slide, please. So, I want to focus now on the problem of getting more granular internet use estimates, and I’m excited to report that last fall NCIA and the Census Bureau announced a new initiative called Local Estimates of Internet Adoption, or Project LEIA for short. With Project LEIA, we’re working to develop more granular single-year estimates using statistical techniques that are so advanced you might just think that they came from a galaxy far, far away. Next slide, please. So, what exactly is Project LEIA? The Census Bureau team is using something called small area estimation to make it possible for us to learn about internet use at the local level. The first product of this effort was released back in September in the form of the first ever experimental single-year estimates of household internet adoption for every county in the United States. For comparison, only about a quarter of U.S. counties have a large enough population to get to internet adoption estimates using the single-year ACS product. And again, the five-year estimates are also able to do this, and they’re very useful, but they’re not ideal in a lot of situations. Next slide, please. The basic idea behind small area estimation is that you take what you already know from survey data and feed that into a model alongside other data points that are known to be linked with the outcome of interest. In this case, we know from previous studies that factors like income and the availability of broadband infrastructure are predictors of internet adoption rates, so we can work with those as auxiliary data sources. Heather will go into more detail about this in a moment, but by incorporating these outside data points alongside the direct survey estimates, you get estimates with smaller margins of error compared with the survey estimates alone, and you also reduce the risk that individual respondents in less populous areas could be re-identified by the publication of new estimates. That makes it feasible to publish reliable estimates for places with smaller populations, all without having to interview any more households or putting anyone’s privacy at risk. Next slide, please. For our first experiment with, you know, with this, the Census Bureau team was able to estimate the proportion of households in every U.S. county that had a wired internet subscription in 2022. We were really excited to be able to share these first results with the world last fall, and we put together an interactive map on NTIA’s website to show off these new estimates. As you can see, there’s quite a lot of variation in adoption rates among U.S. counties, and these estimates enable you to zoom in on a particular area of interest. Next slide, please. While this is a promising start, we think there’s a lot more that can be done with Project LEIA, and NTIA and the Census Bureau are working together on the next steps. At the same time that we released the feasibility study and experimental estimates, NTIA put out an RFC asking for feedback and ideas for future directions. We’ve got some great suggestions that we’ve incorporated into our work moving forward. Right now, we have three tasks that we’re working on as part of this. initiative. First, the Census Bureau team is looking at ways to refine the model for the current set of estimates to get to the point where we all feel ready to shed the experimental label. Second, they are exploring the feasibility of producing even more granular estimates at the census tract level. And as mentioned before, census tracts in the United States are basically neighborhoods in a lot of places, and they generally contain between 1,000 and 8,000 people. And finally, they are working to apply machine learning to bring some of the more detailed computer and internet use data from the NTIA Internet Use Survey to the ACS dataset, opening up new research opportunities and setting the stage for the potential for small area estimates of those more detailed variables in the future. We’re very excited about the potential for project layout to help improve our understanding of internet use in the United States. And I’m pleased to turn next to Heather, who will get into some more of the technical details of how the Census Bureau is making this all possible.

Heather Keene: Thanks, Rafi. I’m happy to continue our 30-year partnership with NTIA, with Project LEIA. And I’m going to reiterate a lot what Rafi said. So Project LEIA is filling in the gap to obtain one-year estimates of internet adoption for all counties, because we need to measure time more quickly than what the five-year ACS estimates can tell us. So the nearest alternative, we could publish the survey estimate from one year from the ACS. But as Rafi and Leslie talked about earlier, you would only get about a quarter of all counties. And so Project LEIA, using small area modeling techniques, allows us to blend that survey with other sources, like Rafi pointed out, to publish internet adoption rates for every single county in the United States. That’s 3,144, I think it was at last count. Okay, so next slide. And so the small area modeling technique that we use, this is a very high-level view of what that looks like. The layout model estimates are really the combination of two separate estimates of internet adoption. And so on the left-hand side of the slide, you see that one component is the AC estimate. This is just from the survey by itself, the proportion of county households that have adopted internet. And on the right-hand side is what we call the indirect estimate. And this is the mechanism that we use to bring in the other related data sources and other factors that are related to internet adoption. So this is a regression model where the outcome is internet adoption at the county level. And the predictors are median household income for the county, the proportion of the population with a bachelor’s degree or higher, and the proportion of county households that have access to broadband infrastructure, which is taken from an administrative source that measures broadband internet infrastructure from another agency. We can talk more about that if you’d like. And so taking these two separate estimates of internet adoption, we blend them together where the contribution of each is determined by the relative precision compared to one another. Next slide, please. So here’s another look at that same visual where we see what happens in larger counties like L.A. County where we have a population of around 10 million people. The ACS direct estimate on proportion of households with internet adoption would be pretty reliable because we would sample a lot of households from this county. Because it’s a very large county, we would send out a lot of questionnaires, and we would get very good data from this county because of its size. And so the final model data estimate, we would expect a higher contribution from this high-quality direct survey estimate where we ask the households directly, do you have access to an internet broadband subscription? And so the LAYA model would draw more information from the ACS compared to the regression model defined in the indirect estimate. Next slide. And so you can imagine for smaller counties like the one in Texas Leslie talked about earlier, we would expect more contribution from the indirect estimate because the ACS estimate for that county, we wouldn’t have a lot of cases to derive an estimate from. And so the resulting estimate of proportion of households would be rather noisy. This would be reflected in the weight that combines the two estimates, the direct estimate from the ACS and the indirect estimate that brings in the additional data. Here, the indirect estimate we would expect to contribute more to the final estimate for that particular county. Okay, so that’s in a nutshell how Project LAYA works. And so the next slide is, this is the same map that Rafi showed you. This is just a screenshot from our feasibility report. Exact same data, but if you want to interact with the data, you’ll go to the NTIA website to look at different views and zoom in and zoom out. So on the right is just kind of a reminder of what exactly are we modeling here. And this is from the actual questionnaire in ACS. So we’re modeling the proportion of all households in each county that subscribe to high speed broadband, internet service, that’s cable, fiber optic, or DSL, just as a reminder. And so this map is a gradient. So darker blue counties are going to be those that have higher proportions of households that have adopted internet with that particular technology type. Next slide. Okay, and so this is just kind of a typical QC that’s done in small area modeling. What we’re looking at here is a scatter plot that compares the proportion of county households that adopt internet compared to that derived from our modeled layer estimate. ACS proportion is on the x-axis or the horizontal, and the layer proportion is on the vertical or y-axis. And so each one of these circles represents the county. And so whenever the layer proportion is exactly the same as the ACS survey proportion, the county or the circle will fall exactly on that diagonal line. So we don’t expect all counties to fall exactly along this line, but generally we want them to follow the general trend. You see some outliers, some counties, some circles that are not very closely aligned to the line. This means that the ACS data doesn’t have a lot of impact here. The model is drawing more from the indirect estimate here. And these are probably going to be very small counties. So this kind of some handful of scatter of points away from a line is completely expected, but we see in general the two estimates agree, and this is as desired. Okay, so overall the layer model progression proportion agrees at a high level with ACS. Okay, next slide. So this is a kind of repeating what Rafi was talking about and what we’re looking to continue Project LEIA. We’re going to fine-tune this county model for all counties for internet adoption and look at some other useful predictors that we didn’t use before that we got from our request for comments from the public. And mainly we’re going to look at measures of urban versus rural. We tried some of those before, but they didn’t work out. So there are other ways that you can measure that. So we’re going to continue exploring that. We’re going to look at measures of economic growth and concentration of the type of jobs or occupations in the county, namely science, technology, engineering, mathematics, or STEM occupations. And so we want to finalize that model, remove the experimental data product label, and have an official data product that measures internet adoption. And what we also want to look at the feasibility of going even further granularity and look at below the county level census tracts Rafi and Leslie described. These are subdivision or neighborhoods of counties around 4,000 people, but these are statistical entities that the Census Bureau creates at the start.

Luis Zambrano Ramos: Thank you so much, Heather, and that concludes our presentation. I’ve put up our website as well as our contact information, and I’m going to pass it over to Susan and Andrew to help us further the discussion.

Susan Chalmers: Thank you so much, Luis. And thank you to everybody for joining us for our open forum today. During this session, we have learned from colleagues at NTIA as well as colleagues from the U.S. Census Bureau on two different surveys. I mean, first we had NTIA’s Internet Use Survey, and then second we heard from the Census Bureau on the American Community Survey. I’d just like to pause to see if anybody has any questions they would like to put forward to our colleagues in person and in Washington, D.C. Sir, please. I believe you may have to use a microphone. You’re welcome to…

Audience: Hi. Do you hear me? Yes. Great. My name is Nenad Dorlich, and I would like to ask, does Census Bureau or NTIA collect any data in regards to sustainability, Internet and sustainability, like electricity consumption from the end user to data centers or CO2 equivalent emissions and such? And the second, I have a second question too, is do you compare your statistic or use data from commercial service provider, FCC, or any other source that may have this information also? Do you use them or compare your findings with them? Thank you.

Susan Chalmers: I’ll just pause to see if colleagues were able to capture the question. Rafi?

Heather Keene: Yeah, I guess for the Census Bureau, yeah, we have lots of different sources that we could potentially use that are collected by other agencies, not specifically the consumption from data centers per se, but we have used in other modeled products data from the Department of Energy, which collects data on energy consumption and detailed. We use that for a project to estimate what proportion of households use air conditioning, which is not asked in the AC. So we brought in that particular data. So the Department of Energy would, we could probably have a look to see what kind of detailed sources about energy consumption are available. And, yes, we do have third-party sources, just depending on the subject matter. The other administrative data that I was talking about that’s used in the LEIA model is data collected from the Federal Communications Commission FCC that measures broadband Internet availability for each county down to the block group level based on speeds that are advertised and the type of technology. So we have a lot of sources that we could potentially draw from. And, yes, we have from other agencies and from commercial sources. Rafi, do you want to add anything to that? Rafi, were you aware of any energy consumption sources that specifically look at data center consumption?

Rafi Goldberg: I am not. I think we would love to get our hands on this data if they were available somewhere. Yeah, I was just going to speak a little bit to the second part of the question. Right. And, you know, and just to add that, you know, in this session we’ve been talking about Internet adoption and usage data, whereas the FCC, as Heather mentioned, is the entity that collects data on where broadband infrastructure is actually available. So we do have that, you know, a whole other data source. And as Heather mentioned, that is actually one of the inputs into the experimental model for Project LEIA.

Audience: Thank you so much. Thank you for the question and thank you for the responses, colleagues. We have a question from an online participant. We’ll go to him next and then, sir, we’ll take your question after the online participant. So, Robert, please, the floor is yours. Yeah, thank you, the presenters. My question is I’m based here in Uganda and I work for a small organization called Youth in Technology and Development Uganda. Yes, I’ve been listening and down here we have a challenge of measuring Internet. I was wondering, do you offer capacity building training now to measure Internet for a small organization like mine, if interested in such?

Jaisha Wray: So I will briefly chime in that that is a very excellent point that we can take back to our colleagues in Washington to consider. We do think that there are great benefits to capacity building on this topic, so more to come. But we work closely with the U.S. Telecommunications Training Institute, and so it’s something to consider for the future. I don’t know if my colleagues online have anything to add.

Susan Chalmers: No, and Robert, we also have provided email addresses on one of those slides, so please let us know if you do not have those email addresses and we can find a way to provide them to you. Thank you for the question. Sir, please.

Audience: Thank you. I’m from RIPE NCC, the regional Internet registry that also operates the largest network of Internet measurement devices, so-called RIPE Atlas props and RIPE Atlas itself. So the question is about, well, when do we talk about the access to the broadband infrastructure? Do we also consider measuring the quality of this infrastructure? Because the access to it is not always the readiness of this infrastructure for the next level of digital economy, for the AI, for all the segmented reality, for this metaverse, et cetera. Do you also measure how do users use this Internet? So is it used only for primary functions or is it used for the access to AI or metaverse applications and activities? Do you think that there should be some measurement done to ensure, and efforts based on that, that should ensure the readiness of Internet infrastructure for the next level of Internet development and digital development? Thank you.

Luis Zambrano Ramos: So I’ll take the first part of the question and then pass it on to my colleagues. So at least for us, the NTIA Internet Use Survey does measure some online activities like social network usage, telework, and the like. We don’t, I think, get to the whole wide range of activities that people can do online, but we’re always looking to improve and iterate upon the survey instrument so that we can actually focus on the way that people are using the Internet today. And, Rafi, I don’t know if you have anything else to add to that.

Rafi Goldberg: Yeah, no, that is exactly right. And, you know, I would just add that in terms of the sort of direct network measurement, you know, there have been efforts at the FCC as well as in the private sector with firms like Ookla and Microsoft. and others to measure actual speeds and latency and other metrics related to the quality of the internet connection the folks actually experience, which I agree is another very important piece of the puzzle. So it’s not one of our programs, but I think another important area for us, certainly.

Audience: Thank you. We’re ready to share our sources as well and our knowledge for that. Thank you.

Susan Chalmers: Wonderful. Let’s let’s exchange information and we can we can be in touch after the session. Are there any other questions? Oh, yes, sir, please.

Audience: Thank you. I am I audible? All right. This is Joshua from Uganda, ISOC chapter. I have a question on the disaggregation. Do you do any disaggregation to maybe like internet in schools? Because I think you seem to focus on households. What about maybe in schools and in other institutions? Is that something you also measure? And how, if you do so? Thank you.

Susan Chalmers: Thank you. Excellent question made turn to our our colleagues in Washington.

Rafi Goldberg: Sure, I can start. So these are household surveys, which is which is why, you know, I think you’ve been hearing a lot about households. We do in the NTIA Internet use survey ask a range of questions about different locations of Internet use. And one of those is Internet use at school. So we do capture some information about that through this survey. I understand that there are other agencies throughout the government that also direct surveys towards the schools themselves. And through that, right. If you’re interested in learning more about how the school itself is using the Internet, those those would probably be the, you know, places to go for that information. But we do ask about Internet use at school, as well as using the Internet for online courses or training is one of the many activities that we ask about as well.

Susan Chalmers: Excellent, thank you. Anything to add from us on the census side?

Leslie Davis: Yes, hi. So, yeah, Rafi is exactly right on the ACS side. I think we’re a little bit restricted in terms of how the questions are mandated. And as a result, the questions are at the highest household level. But there are other sources, I believe, within the census as well on other surveys that get more at the school level or how trainings are done at the school level.

Luis Zambrano Ramos: And actually, one thing I want to plug in is one of, I believe, Leslie or Heather’s slides, it’s data.census.gov for access to various census data products.

Susan Chalmers: Thank you, just checking if we have any questions.

Andrew Flavin: Nothing more online.

Susan Chalmers: OK, any any questions from the room? Please.

Audience: So a quick one from international think tank perspective. When we speak to member states, the trouble is not just data collection, but actually using it for targeted initiatives for those who are not yet included or using the Internet. So how does your data collection feed in to the decision-making process at county or state or central federal government level to really make initiatives that target those geographical segments and those household and user segments that are not yet got the access, got the skills, or not yet using the Internet for various functions, anything from, you know, participating in public discourse, using government services online, shopping online, banking online. How is your data collection feeding into that to really ensure that we get those who are excluded online and included? Thanks.

Susan Chalmers: Thank you for the question. May I turn to Rafi, would you be able to offer a response to the question?

Rafi Goldberg: Yeah, you know, I think, you know, I have heard from many state broadband offices in particular that, you know, they like to make use of our data when, you know, they’re designing programs to, you know, serve their their constituents. So certainly, you know, at the federal level, we don’t have a monopoly on how we use these data, you know, certainly state and local governments and nonprofits and the private sector make use of these data all the time for their own work. You know, and, you know, it is not uncommon for us here at NCIA and elsewhere in the government to cite some of our data, for example, in an FCC filing or you know, in another venue where, you know, we are, you know, analyzing various policy issues. So, you know, so that I think is, you know, part of how this is helpful. But, you know, we certainly don’t think that we have a monopoly on, you know, how to best make use of the data. One very important feature of these data sets is they’re all available for public use and researchers and advocates on the outside do a lot of great things using our data.

Susan Chalmers: Thank you so much, Rafi. We just have a few minutes left. I would just like to invite the audience to share any of their experiences on measuring Internet use. Please feel free. We did flag these questions towards the top of the session, but we still do have some time if anybody would like to contribute.

Audience: Well, I can share a bit about our experience not about sharing Internet or measuring Internet use, but with the initiative that is called Internet Measurement Day that we do together with ICANN and we do it in different countries going there and just trying to show the country how to use the Internet measurement instruments to see what is their counter position on routing security, what is their counter position on the interconnection with other countries, what is their counter condition about the peering inside the country? So I guess we should have similar activities so we can have more understanding how we can improve the situation to have these networks ready because we talk a lot about these digital services, but there should be also the understanding that to ensure these digital services, to ensure this next level to digital development, we should have an appropriate level of Internet development in the country, appropriate level of critical resilient infrastructures in the country, route servers, IXPs, all the stuff that is connected and that is equivalent to the needs of digital economy and society of the future. Of now, because now is the future.

Susan Chalmers: Thank you, thank you so much. We are wrapping up and our time is concluding, so I just want to thank everybody for their contributions to this discussion and Jayshia, may I turn it to you to conclude?

Jaisha Wray: Sure. Again, thank you for participating. We really appreciated the active discussion. It’s clear there is a lot of interest in this topic. Our goal today was to introduce it and then to continue the discussion. So please feel free to reach out to us. Our virtual doors are open and we look forward to hearing from all of you and continuing to exchange lessons learned in this area and moving forward and exploring areas for cooperation as well. So again, thank you and we’ll see you next time. Take care. Thank you.

Agreement points

Need for comprehensive Internet measurement data to inform policy decisions

Speakers

– Luis Zambrano Ramos
– Leslie Davis
– Rafi Goldberg
– Heather Keene

Arguments

NTIA Internet Use Survey as longest-running federal data collection on computer and Internet use since 1994

ACS provides nationally representative data on Internet adoption and computer ownership for 3.5 million households annually

State broadband offices utilize survey data for designing targeted programs to serve constituents

Small area estimation technique combines survey data with auxiliary predictors like income and broadband infrastructure availability

Summary

All speakers from NTIA and Census Bureau agree that robust data collection is essential for understanding Internet adoption patterns and informing evidence-based policy decisions at federal, state, and local levels.

Topics

Development | Infrastructure | Legal and regulatory

Geographic granularity limitations in current survey methodologies

Speakers

– Leslie Davis
– Rafi Goldberg
– Heather Keene

Arguments

ACS one-year data limited to counties and places with 65,000+ population, while five-year data reaches census tract level

Current surveys unable to provide reliable estimates for smaller geographic areas due to sample size constraints

Project LEIA produces first-ever experimental single-year estimates of household Internet adoption for every U.S. county

Summary

Speakers acknowledge that traditional survey methods face limitations in providing reliable estimates for smaller geographic areas, necessitating innovative approaches like Project LEIA to fill these gaps.

Topics

Development | Infrastructure

Importance of data accessibility and transparency for research and policy development

Speakers

– Luis Zambrano Ramos
– Leslie Davis
– Rafi Goldberg

Arguments

Public availability of raw data through GitHub repository and explorer tools for researchers and policymakers

ACS data enables analysis of broadband adoption disparities at sub-state geographic levels for policy development

State broadband offices utilize survey data for designing targeted programs to serve constituents

Summary

All speakers emphasize the critical importance of making Internet measurement data publicly available and accessible to enable research, policy analysis, and program development by various stakeholders.

Topics

Development | Legal and regulatory

Similar viewpoints

Both speakers emphasize the importance of iterative improvement in survey methodology through stakeholder engagement and continuous refinement to enhance data quality and relevance.

Speakers

– Luis Zambrano Ramos
– Leslie Davis

Arguments

Public comment process for survey instrument development ensuring stakeholder input in data collection design

Survey methodology improvements in 2016 to address question clarity and response reliability issues

Topics

Development | Legal and regulatory

Both speakers advocate for advanced statistical techniques to overcome traditional survey limitations and provide more granular, timely data for local-level policy and program evaluation.

Speakers

– Rafi Goldberg
– Heather Keene

Arguments

Need for more timely local-level data to understand program impacts and community-specific Internet use patterns

Project LEIA produces first-ever experimental single-year estimates of household Internet adoption for every U.S. county

Small area estimation technique combines survey data with auxiliary predictors like income and broadband infrastructure availability

Topics

Development | Infrastructure

Both speakers recognize the value of international cooperation and knowledge sharing to advance Internet measurement capabilities globally and learn from different countries’ experiences.

Speakers

– Jaisha Wray
– Susan Chalmers

Arguments

International collaboration needed to advance understanding of Internet usage measurement globally

Coordination opportunities with international measurement initiatives and infrastructure assessment programs

Topics

Development | Infrastructure

Unexpected consensus

Recognition of survey methodology challenges and limitations

Speakers

– Rafi Goldberg
– Leslie Davis
– Heather Keene

Arguments

Declining survey response rates and evolving technology terminology requiring constant survey instrument updates

Survey methodology improvements in 2016 to address question clarity and response reliability issues

Current surveys unable to provide reliable estimates for smaller geographic areas due to sample size constraints

Explanation

It is somewhat unexpected that government agencies would openly acknowledge the limitations and challenges in their own data collection methods. This transparency demonstrates a commitment to continuous improvement and honest assessment of methodological constraints.

Topics

Development | Infrastructure

Openness to international capacity building and knowledge sharing

Speakers

– Jaisha Wray
– Susan Chalmers
– Audience

Arguments

Potential for capacity building partnerships through U.S. Telecommunications Training Institute

Request for capacity building training on Internet measurement from organizations in developing countries like Uganda

Coordination opportunities with international measurement initiatives and infrastructure assessment programs

Explanation

The willingness of U.S. government agencies to share expertise and collaborate internationally on Internet measurement, especially with developing countries, represents an unexpectedly collaborative approach that goes beyond typical domestic policy focus.

Topics

Development | Capacity development

Overall assessment

Summary

The discussion reveals strong consensus among speakers on the fundamental importance of comprehensive Internet measurement data, the need for methodological innovation to address geographic limitations, and the value of data transparency and accessibility. There is also agreement on the challenges facing current survey methodologies and the potential for international collaboration.

Consensus level

High level of consensus with collaborative implications – The speakers demonstrate remarkable alignment on both the challenges and opportunities in Internet measurement. This consensus suggests strong potential for continued inter-agency cooperation, methodological advancement through projects like LEIA, and international knowledge sharing. The agreement extends beyond technical issues to include policy applications and capacity building, indicating a mature and coordinated approach to addressing digital divide measurement challenges.

Different viewpoints

Unexpected differences

Scope of Internet measurement beyond adoption statistics

Speakers

– Luis Zambrano Ramos
– Rafi Goldberg
– Audience

Arguments

NTIA Internet Use Survey measures device usage, online activities, and Internet access technologies through CPS supplement

State broadband offices utilize survey data for designing targeted programs to serve constituents

Interest in measuring Internet infrastructure quality and readiness for next-generation digital applications beyond basic access

Explanation

While NTIA representatives focused on measuring adoption, usage patterns, and basic online activities, audience members raised concerns about measuring infrastructure quality, sustainability impacts (energy consumption, CO2 emissions), and readiness for advanced applications. This revealed an unexpected gap between what government agencies currently measure versus what stakeholders believe should be measured for comprehensive Internet assessment.

Topics

Infrastructure | Development

Overall assessment

Summary

The discussion revealed minimal direct disagreements among speakers, with most participants sharing common goals around improving Internet measurement capabilities. The main areas of divergence centered on measurement scope and implementation approaches rather than fundamental disagreements about objectives.

Disagreement level

Low level of disagreement with high consensus on core objectives. The implications are positive for the field, as stakeholders share common ground on the importance of Internet measurement while bringing complementary perspectives on how to enhance current capabilities. The discussion suggests opportunities for expanding measurement frameworks to address broader stakeholder needs while building on existing statistical infrastructure.

Partial agreements

Similar viewpoints

Both speakers emphasize the importance of iterative improvement in survey methodology through stakeholder engagement and continuous refinement to enhance data quality and relevance.

Speakers

– Luis Zambrano Ramos
– Leslie Davis

Arguments

Public comment process for survey instrument development ensuring stakeholder input in data collection design

Survey methodology improvements in 2016 to address question clarity and response reliability issues

Topics

Development | Legal and regulatory

Both speakers advocate for advanced statistical techniques to overcome traditional survey limitations and provide more granular, timely data for local-level policy and program evaluation.

Speakers

– Rafi Goldberg
– Heather Keene

Arguments

Need for more timely local-level data to understand program impacts and community-specific Internet use patterns

Project LEIA produces first-ever experimental single-year estimates of household Internet adoption for every U.S. county

Small area estimation technique combines survey data with auxiliary predictors like income and broadband infrastructure availability

Topics

Development | Infrastructure

Both speakers recognize the value of international cooperation and knowledge sharing to advance Internet measurement capabilities globally and learn from different countries’ experiences.

Speakers

– Jaisha Wray
– Susan Chalmers

Arguments

International collaboration needed to advance understanding of Internet usage measurement globally

Coordination opportunities with international measurement initiatives and infrastructure assessment programs

Topics

Development | Infrastructure

Key takeaways

The digital divide remains a critical economic and civil rights issue, with NTIA and Census Bureau providing essential data through two major surveys: the NTIA Internet Use Survey (running since 1994) and the American Community Survey (ACS)

Current survey methods have significant limitations in providing granular geographic data, particularly for smaller communities and rural areas, due to sample size constraints

Project LEIA represents a breakthrough innovation using small area estimation techniques to produce single-year internet adoption estimates for every U.S. county, combining survey data with auxiliary predictors like income and infrastructure availability

There is strong international interest in capacity building for internet measurement, with developing countries seeking training and technical assistance

Data quality and infrastructure readiness are emerging concerns, with stakeholders emphasizing the need to measure not just access but the quality and capability of internet infrastructure for next-generation applications

The surveys serve as vital tools for policymakers at federal, state, and local levels to design targeted programs addressing digital inclusion gaps

Resolutions and action items

NTIA to consider capacity building training opportunities for international organizations, potentially through collaboration with the U.S. Telecommunications Training Institute

Project LEIA team to continue refining the county-level model to remove the ‘experimental’ label and produce official data products

Census Bureau to explore feasibility of producing even more granular estimates at the census tract (neighborhood) level

Research team to apply machine learning techniques to bring detailed internet use data from NTIA surveys to ACS datasets

Incorporation of additional predictors into LEIA model including urban/rural measures and STEM occupation concentrations based on public feedback

Exchange of contact information and data sources between NTIA/Census Bureau and international measurement organizations like RIPE NCC

Unresolved issues

How to effectively measure internet infrastructure quality and readiness for next-generation applications like AI and metaverse beyond basic access metrics

Addressing declining survey response rates across both major data collection efforts

Balancing the need to update survey instruments for evolving technologies while maintaining time series comparability

Measuring internet sustainability impacts including electricity consumption and CO2 emissions from end users to data centers

Expanding measurement beyond households to include schools and other institutional settings more comprehensively

Developing more effective mechanisms to translate data collection into targeted policy interventions for digitally excluded populations

Suggested compromises

Using five-year ACS estimates for smaller geographic areas when single-year data is not reliable, accepting the trade-off between timeliness and geographic granularity

Leveraging multiple data sources (survey data, administrative records, commercial data) through small area estimation techniques to balance precision and coverage

Collaborating with other federal agencies (FCC, Department of Energy) and private sector entities to supplement survey data with infrastructure and quality measurements

Maintaining both detailed NTIA Internet Use Survey for comprehensive activity measurement and streamlined ACS questions for broad geographic coverage

Does Census Bureau or NTIA collect any data in regards to sustainability, Internet and sustainability, like electricity consumption from the end user to data centers or CO2 equivalent emissions and such?

Speaker

Nenad Dorlich

Reason

This comment was insightful because it introduced an entirely new dimension to internet measurement that hadn’t been considered in the presentation – the environmental impact of internet usage. It challenged the traditional focus on access and adoption by highlighting the sustainability implications of digital connectivity, which is increasingly relevant given climate concerns and the growing energy consumption of data centers.

Impact

This question shifted the discussion from purely social and economic metrics to environmental considerations. It prompted the speakers to acknowledge gaps in their current data collection and opened up a new avenue for future research. The response revealed that while they don’t currently collect this type of data, they are open to exploring it, demonstrating how external perspectives can identify blind spots in existing measurement frameworks.

Do we also consider measuring the quality of this infrastructure? Because the access to it is not always the readiness of this infrastructure for the next level of digital economy, for the AI, for all the segmented reality, for this metaverse, et cetera.

Speaker

RIPE NCC representative

Reason

This comment was particularly thought-provoking because it challenged the fundamental assumption that measuring access equals measuring readiness for digital participation. It introduced the critical distinction between having internet access and having quality internet that can support emerging technologies and advanced digital economy activities.

Impact

This comment deepened the technical discussion by highlighting the inadequacy of binary access measurements. It pushed the conversation toward more nuanced quality metrics and future-readiness assessments. The speakers acknowledged this as an important gap, referencing other organizations’ efforts to measure actual speeds and latency, which showed how this comment helped identify complementary measurement approaches.

The trouble is not just data collection, but actually using it for targeted initiatives for those who are not yet included or using the Internet… How is your data collection feeding into that to really ensure that we get those who are excluded online and included?

Speaker

International think tank representative

Reason

This comment was insightful because it shifted focus from the technical aspects of data collection to the practical application and policy impact of the data. It challenged the presenters to consider whether their sophisticated measurement systems actually translate into effective interventions for digital inclusion.

Impact

This question fundamentally changed the discussion’s trajectory from methodology to impact assessment. It forced the speakers to address the gap between data collection and policy implementation, leading to acknowledgment that while they provide data, they don’t control how it’s used for targeted interventions. This highlighted a critical weakness in the data-to-action pipeline and sparked discussion about the need for better coordination between data collectors and program implementers.

Do you do any disaggregation to maybe like internet in schools? Because I think you seem to focus on households. What about maybe in schools and in other institutions?

Speaker

Joshua from Uganda ISOC chapter

Reason

This comment was valuable because it exposed a significant limitation in the measurement approach – the heavy focus on household-level data while potentially missing institutional access points that are crucial for digital inclusion, especially in developing contexts where schools and public institutions may be primary internet access points.

Impact

This question highlighted a methodological blind spot and prompted discussion about the limitations of household-focused surveys. It led to acknowledgment that other agencies collect school-level data, revealing the fragmented nature of internet measurement across different sectors and the need for more comprehensive approaches that capture institutional access.

Overall assessment

These key comments fundamentally transformed what began as a technical presentation about measurement methodologies into a more critical examination of the limitations and gaps in current approaches to measuring internet use. The questions from international participants were particularly impactful, as they brought perspectives from different contexts and challenged US-centric assumptions. The comments collectively pushed the discussion beyond the ‘how’ of measurement to address the ‘why’ and ‘what’s missing’ questions. They revealed that while the US has sophisticated measurement systems, there are significant gaps in environmental impact assessment, quality measurement, institutional access tracking, and most importantly, in translating data into effective policy interventions. The international perspective was crucial in highlighting these limitations and suggesting that comprehensive internet measurement requires a more holistic approach that considers sustainability, quality, diverse access points, and practical policy applications.

Does Census Bureau or NTIA collect any data regarding sustainability, Internet and sustainability, like electricity consumption from end user to data centers or CO2 equivalent emissions?

Speaker

Nenad Dorlich

Explanation

This identifies a gap in current data collection around environmental impact of internet usage, which is increasingly important for policy making

Do you offer capacity building training to measure Internet for small organizations?

Speaker

Robert (online participant from Uganda)

Explanation

This highlights the need for international capacity building and knowledge transfer on internet measurement methodologies

Do you measure the quality of broadband infrastructure and its readiness for next-level digital economy applications like AI, metaverse, etc.?

Speaker

RIPE NCC representative

Explanation

This addresses the gap between measuring access versus measuring quality and capability of internet infrastructure for emerging technologies

Do you measure how users actually use the Internet – whether for primary functions or advanced applications like AI or metaverse?

Speaker

RIPE NCC representative

Explanation

This explores the need to understand not just access but the sophistication of internet usage patterns

Do you do disaggregation for internet use in schools and other institutions beyond households?

Speaker

Joshua from Uganda ISOC chapter

Explanation

This identifies a limitation in current surveys that focus primarily on household data rather than institutional usage

How does data collection feed into decision-making processes for targeted initiatives to include those not yet online?

Speaker

International think tank representative

Explanation

This addresses the critical gap between data collection and practical policy implementation for digital inclusion

Exploration of urban versus rural measures and other predictors for the LEIA model refinement

Speaker

Heather Keene

Explanation

This is ongoing research to improve the accuracy of small area estimation models for internet adoption

Feasibility of producing census tract level estimates (neighborhood level) for internet adoption

Speaker

Rafi Goldberg/Heather Keene

Explanation

This would provide even more granular geographic data for local policy making and program targeting

Application of machine learning to bring detailed internet use data from NTIA survey to ACS dataset

Speaker

Rafi Goldberg

Explanation

This could significantly expand the types of internet usage data available at local levels for research and policy