Heartbeat Analysis: Methodology FAQ

Heartbeat Analysis is OrgAcuity's method of identifying what matters most to each individual respondent — surfacing items they rated meaningfully higher or lower than their personal average. This article addresses common methodology questions about minimum item requirements, missing data handling, and threshold logic.

How many items are needed for reliable Heartbeat Analysis?

We recommend a minimum of five rating items for Heartbeat Analysis to produce meaningful results. This threshold aligns intentionally with OrgAcuity's standard confidentiality threshold. Since n = 5 is the most common confidentiality threshold for surfacing quantitative results, managers see strengths and opportunities for their team if they have enough respondents to view results. 

A note on driver analysis: Heartbeat Analysis is distinct from driver analysis. OrgAcuity uses Kendall's τ-b for driver analyses given the ordinal scale of the rating data. While there's no universal consensus on minimum sample size for this approach, OrgAcuity requires n ≥ 25 to support reliable estimates. This means driver analysis typically cannot be run at the manager level; it requires a sufficiently large aggregation. This is meaningfully different from how some platforms handle manager-level "engagement drivers," which may reflect data aggregated beyond the manager's direct team rather than that team's own results.

How is missing data handled?

If a respondent skips a survey item, that item is excluded from their individual results. OrgAcuity does not impute missing responses.

For Heartbeat Analysis, the impact works as follows: a respondent's within-person mean and standard deviation are calculated using only the items they answered. If a respondent did not answer a particular item, they are excluded from the vote aggregation for that item. This preserves the integrity of the individual baseline by ensuring it's built only from items the respondent actually rated.

The practical implication: items with higher skip rates may have fewer respondents contributing to their vote totals, which is worth keeping in mind when interpreting strengths or opportunities with lower response counts.

How is the up-vote / down-vote threshold determined?

An item receives an up-vote if a respondent rates it more than one within-person standard deviation above their individual mean. It receives a down-vote if rated more than one within-person SD below their mean. Items within one SD in either direction are treated as neutral — neither a clear strength nor a clear opportunity for that respondent.

The one-SD threshold is not derived from inferential statistics. Heartbeat Analysis is not estimating population parameters from a sample; it's characterizing an individual respondent's response pattern. In that context, one standard deviation serves as a practical, interpretable signal of meaningful deviation from someone's personal baseline, distinct from statistical significance. The goal is to identify what genuinely stands out for each person, not to test a hypothesis.

This design also makes the methodology robust at small group sizes, where traditional significance testing would be unreliable.

Don't up-votes and down-votes correlate with higher and lower average scores?

Here's a concrete example that shows how average scores and heartbeat votes can diverge sharply:

Scenario: "Recognition" item at a mid-size tech company

What's happening beneath the surface

Team A's 3.90 on recognition reflects a consensus. Nearly all respondents rated it a 4 on a 5-point scale. Two rated it a 3. There's almost no disagreement. The heartbeat confirms what the average implies: 90% of the team is voting up. A manager looking at this result can feel confident that recognition is landing broadly and consistently.

Team B's 3.80 on recognition is a statistical illusion. Seven employees rated it a 5 — they feel genuinely recognized, likely by a manager who invests heavily in a small inner circle. Eleven employees rated it a 3, a below-midpoint score that signals quiet dissatisfaction rather than outright disengagement. Two employees rated it a 4. Those three groups average out to 3.80, nearly identical to Team A. But the heartbeat tells the real story: 55% of the team is voting down.

Heartbeat flags Team B as a divided team, not a mediocre one. That distinction matters. Mediocre teams need encouragement. Divided teams need diagnosis. The right intervention here is understanding who feels unrecognized, whether it maps to tenure, role, or network position, and whether the manager is inadvertently creating in-group/out-group dynamics.

Integrating organizational network data

This is where layering in ONA can provide significant value. If Team B's dissatisfied employees are also peripheral in the network — low connectedness, few mutual connections with the manager — the heartbeat pattern isn't just a survey artifact. It's a signal of structural exclusion. The network data explains why the polarization exists in a way that averages never could.

Average scores tell you what. Heartbeat tells you how. ONA tells you why.

Have additional questions about Heartbeat Analysis or OrgAcuity's methodology? Contact your Customer Success Manager or reach out at support@orgacuity.com.