Oak Ridge National Laboratory · AI INTEGRATION PROGRAM

How might we measure how an org perceives & uses AI over time?

Most organizations measure AI adoption by counting licenses. I built Oak Ridge National Laboratory a longitudinal survey program that measures what staff know, what they use, what worries them, and whether training keeps pace with encouragement. Four waves in, its findings led directly to two projects I then led: an overhaul of the org's AI and data-use guidance, and a redesign of its internal AI website.

BY THE NUMBERS

  • 1,000+survey responses from staff
  • 4archetypes stable across waves
  • 4waves over 2 years
DATE
Winter 2024 -
ORGANIZATION
Oak Ridge National Laboratory
ROLE
Research leadSurvey designLongitudinal analysisSegmentationAI governance
DELIVERABLES
Longitudinal AI awareness survey program, user archetype segmentation model, best-worst scaling analysis of tool-selection priorities, Power BI dashboard for longitudinal trends, cross-study synthesis and recommendation set for leadership

How was the program developed?

  1. 01

    Design with decision-makers

    I built the survey with the AI for Operations program lead and the AI Governance Committee. Every question maps to a decision leadership has to make: awareness, use, risk perception, and outlook. That is why the results get used instead of filed.

  2. 02

    Segment the workforce

    The survey uses validated scales where they exist. From the first wave, I used cluster analysis to build four archetypes: Enthusiasts, Explorers, Wary Experts, and Skeptics. An org-wide average can't tell you who needs training and who needs permission; segments can. The model has held across all four waves.

  3. 03

    Track changes over time

    Each wave samples across every major directorate. Between waves two and three, weekly AI use rose from 48% to 75% of staff, and Enthusiasts grew from 40% to 54%. The gap between encouragement to use AI and training for it widened from 12 to 26 points.

  4. 04

    Adding methods as questions change

    Once most staff used AI weekly, the question became which tools and why. Best-worst scaling showed speed and reliability drive tool choice, while access to the newest models matters least, which challenged how enterprise tools were being justified. Coding the open text separated two risk stories, fear of wrong answers and fear of leaked data, so my recommendations split the same way.

Leadership gets a repeatable read on where AI adoption is speeding up, where it's stalling, and why.

Our research identified 4 distinct AI user archetypes with significant differences in AI work practices and overall perspectives on AI:

  • Enthusiasts

    AI-powered professionals

    • Use AI at work frequently
    • High AI awareness
    • Positive outlook about AI
  • Explorers

    Informed AI adopters

    • Use AI at work sometimes
    • High AI awareness
    • Positive outlook about AI
  • Wary Experts

    Concerned AI specialists

    • Don't really use AI at work
    • High AI awareness
    • Negative outlook about AI
  • Skeptics

    AI-resistant questioners

    • Don't really use AI at work
    • Low AI awareness
    • Negative outlook about AI

Why? Grouping respondents into archetypes allows us to better perceive and strategically tailor interventions to meet specific workflows, needs, and motivations, ultimately driving greater AI adoption, understanding, and efficiency.

How? Responses to a series of questions about frequency of AI use at work, AI awareness, and overall AI outlook were clustered into AI user archetype groupings.

The findings led directly to my overhaul of the org's AI and data-use guidance and my redesign of its internal AI website. They also shaped training content, tooling priorities, and the case for funding AI enablement as an enterprise capability. Detailed findings stay internal.

Designing a longitudinal measurement system and sustaining it across multiple waves is one skill - extending it with methods matched to a maturing question and synthesizing it against additional data into a single argument for leadership, is a different one. Together they demonstrate that I can build the evidence base an organization needs to make AI decisions and translate it into strategy cross-functional leaders will act on.