Metric Registry
This page is the working reference for Bloomfilter's metrics It catalogs all current metrics, the signals that indicate trouble, the actions to take, and the business questions they answer.
Two ways to use it:
You see a metric in the platform — search this page for its name (e.g. Review Time) to get its definition, what "bad" looks like, and what to do about it.
You have a problem or use case — start from Find a metric fast or the Use case library below to find which metrics support it.
Tip: use your browser find (Ctrl/Cmd-F) or GitBook AI to jump straight to a metric, a symptom, or a role.
How this registry is organized
Glossary
Definitions on how a metric is calculated
Find a metric fast
Symptom-to-metric and role-to-metric lookups
Personas
The 5 roles Bloomfilter serves, in their own words
Executive priorities
The 9 business priorities and the metrics that prove them
Metric reference
All 65 metrics: definition, signals, actions, and owner
Use case library
29 common jobs-to-be-done and the metrics behind them
Financial impact
How metrics translate into cost savings and revenue
Glossary
Backward Steps
Percentage of tasks that moved backward in the workflow.
Bulk Status Changes
Number of tasks that changed status in bulk for a given status type.
Complexity
Average story points of work units within the selected scope.
Cost
Determined by the data entered in monthly inputs for the selected
scope.
Count
Number of tasks moving through the process.
Cycle Time
Average amount of time it takes for a task to move from start to
completion.
Cycle Time (days)
Average number of days it takes a task to move through the
workflow.
Cycle Time Inconsistency
Degree of variation in cycle time across tasks within a work period.
Declined Change Requests
Number of change requests that were rejected.
Defect Ratio Measure
Ratio of defect-related tasks compared to total completed work.
Flow of Work by Phase
Distribution of work as it moves through different workflow phases.
Goals
Target outcomes defined for a team or work period.
Independence
Degree to which work items progress without external
dependencies.
Initiative Completion
Percentage of initiatives that have been completed.
Initiative Completion (Points)
Percentage of total story points associated with an initiative that
have been completed.
Initiative Completion (Tasks)
Percentage or count of initiative-related tasks that have been
completed.
Initiative Focus
Measure of how concentrated effort is on active initiatives.
Initiative Focus (Points)
Measure of how many completed story points in the selected
scope belong to a specific initiative.
Initiative Focus (Tasks)
Measure of how many completed tasks in the selected scope
belong to a specific initiative.
Initiative Overview
High-level summary of an initiative's current status, progress, and
key delivery metrics.
Key Measures Over Time
Referenced as a standalone metric category in the sheet (covers Readiness, Review Time, Scope, Strategy over time)
Lead Time
The average time of transition between status type to do and status type done
Missing Days
The number of days when no recorded work happened
Multiple Point Changes per Task
Count of tasks where the point estimate changed more than once
Planned vs Actual Work
Comparison between planned work and work actually completed.
Points Acceptance Rate
Percentage of committed story points completed by the end of the
sprint.
Pull Request Closed %
Percentage of pull requests that are closed, highlighting rejected
or incomplete changes.
Quality
The number of tech debt and bugs divided by the total number of tasks within the selected date range
Readiness
Percentage of tasks that meet predefined criteria to be considered ready for development
Readiness (Key Measures Over Time)
Percentage of tasks in the selected scope that are ready for
development.
Readiness (Sprint)
Percentage of sprint tasks that meet readiness criteria.
Reaction Time
Average time it takes for tasks to move from backlog into active
development after creation or commitment.
Review Time
Average calendar time between when a change request is opened
and when it is closed.
Review Time (Key Measures Over Time)
Average duration of code review from open to completion.
Scope
Monitors changes to sprint/project scope after initial commitment; percentage of scope added or removed within the selected date range
Scope Change
Measure of tasks added or removed after the start of a work period.
Scope (Key Measures Over Time)
Changes to sprint or project scope after initial commitment.
Sprint Goals
Objectives defined for the sprint that guide planning and evaluation.
Sprint Scope Creep
Percentage change in sprint scope after the sprint starts.
Sprint Scope Volatility
Total absolute change in story points during a sprint.
Stale Tasks
Number of tasks that have not changed status for a defined period.
Strategy
Percentage of tasks associated with an epic within the selected
date range.
Strategy (Key Measures Over Time)
Percentage of tasks tied to an epic within the selected scope.
Task Acceptance Rate
Percentage of committed sprint tasks completed by sprint end.
Table View – Contributing Boards to an Initiative
Boards contributing work to an initiative.
Table View – Contributing Teams
Teams contributing work units or points to an initiative.
Table View – Costs
Cost summary of each epic within an initiative.
Table View – Flow By Phase (Initiative Epics)
Progress of initiative epics through workflow phases.
Table View – Related Work Units
Related stories, bugs, and tasks tied to an initiative.
Table View – Throughput (Initiative – Points)
Initiative completion measured in story points.
Table View – Throughput (Initiative – Tasks)
Initiative completion measured in tasks.
Table View – Throughput (Contributing Epics)
Epics contributing work to an initiative.
Table View – Throughput (Contributing Teams)
Teams contributing work to an initiative.
Technical Debt Tasks
Work items labeled as technical debt.
Tasks Moving Backwards
Percentage of tasks moving to an earlier workflow state.
Test Coverage
Percentage of work items that include test cases.
Throughput
Delivery rate measured by completed tasks over time.
Tree Map View – Cost by Initiative
Visual cost contribution by initiative epics.
Untested Tasks
Percentage of completed tasks that were not tested.
Velocity
Amount of work completed in points during a time period.
Waiting
Percentage of tasks entering a waiting status.
Work Pace Explorer
Visualization comparing planned vs actual delivery.
Work Period Performance Score
Daily comparison of expected vs actual delivery.
Work Plan Inconsistency
Variation in estimated effort across work periods.
Find a metric fast
By symptom
When you hear one of these from a stakeholder, these are the metrics to pull up.
"We're always behind schedule"
Lead Time · Cycle Time · Initiative Slippage Cost
"Quality is suffering"
Escaped Defect Rate · Untested Tasks · Quality Trend
"Not sure if AI is helping"
All AI metrics (Agent Adoption through Planning vs Execution Split)
"Teams seem busy but not shipping"
Throughput · Stale Tasks · Strategy Alignment
"Builds keep breaking"
CI Stability · Failure Rate · Rerun Rate
"Can't tell if vendors are worth it"
Vendor Performance · Cost per Output
"Engineering is too expensive to justify"
Cost by Initiative · Rework Cost · PMO Cost · Cost per Output
By role
CTO / VP Engineering
Velocity, quality, team health, strategic alignment, AI ROI
"We're always behind schedule."
All 65 metrics
CFO / Finance
Cost control, waste reduction, vendor optimization, AI spend visibility
"Engineering is expensive and I can't justify the budget."
Cost by Initiative, AI Spend by Task/Initiative, Rework Cost, QA Failure Cost, Production Defect Cost, PMO Cost, Initiative Slippage Cost, Vendor Performance, Cost per Output
CPO / Product
Roadmap predictability, scope management, alignment to strategy
"Engineering keeps missing dates."
Work Mix, Scope Creep, Work-plan Inconsistency, Strategy Alignment, Cost by Initiative, Strategic Scope Creep, Initiative Slippage Cost
Delivery Lead / Scrum Master
Sprint predictability, blockers, process efficiency, team capacity
"Sprints feel chaotic."
Cycle Time, Reaction Time, Work-plan Inconsistency, Bulk Status Changes, Velocity, Sprint Predictability, Flow by Pace, Collaboration Intensity, Responsiveness
Engineering Manager
Code review speed, CI stability, rework reduction, collaboration health
"Developers are waiting too long for reviews."
Tasks Moving Backwards, Collaboration Intensity, Responsiveness, Review Time, Declined Change Requests, Merge Rate, PR Throughput, Draft-to-Merge Lag, Review Depth, Reviewer Participation, CI Stability, Failure Rate, Rerun Rate, CI Duration, Untested Tasks, Quality Trend, Failure Hotspots, Planning vs Execution Split
Personas
The five roles Bloomfilter is built for. Partners will most often be helping a CTO/VP Engineering or a CFO.
CTO / VP Engineering
Overall engineering effectiveness and business impact
Primary concerns — Velocity, quality, team health, strategic alignment, AI ROI
Questions they ask — Are we building the right things? Are we moving fast enough? What's our quality like? Are teams healthy? Is AI paying off?
How they describe problems — We're always behind schedule. Teams seem busy but we're not shipping fast enough. I can't tell if we're working on the right things. Is AI helping or just costing money?
Metrics they care about most — All 65 metrics
CFO / Finance
Engineering ROI, cost management, budget efficiency
Primary concerns — Cost control, waste reduction, vendor optimization, AI spend visibility
Questions they ask — What are we spending? What's the cost per feature? Where is money being wasted? Is engineering a good investment? What's AI costing us?
How they describe problems — Engineering is expensive and I can't justify the budget. We're spending millions on rework. I have no idea if AI is worth it. Vendors are billing but where's the value?
Metrics they care about most — Cost by Initiative, AI Spend by Task/Initiative, Rework Cost, QA Failure Cost, Production Defect Cost, PMO Cost, Initiative Slippage Cost, Vendor Performance, Cost per Output
CPO / Product
Product roadmap delivery, strategic outcomes, customer value
Primary concerns — Roadmap predictability, scope management, alignment to strategy
Questions they ask — Are we delivering on roadmap commitments? Why do initiatives keep slipping? Is engineering focused on what matters?
How they describe problems — Engineering keeps missing dates. Scope keeps changing mid-flight. I don't know if teams are working on high-priority features or random stuff.
Metrics they care about most — Work Mix, Scope Creep, Work-plan Inconsistency, Strategy Alignment, Cost by Initiative, Strategic Scope Creep, Initiative Slippage Cost
Delivery Lead / Scrum Master
Sprint execution, flow optimization, team coordination
Primary concerns — Sprint predictability, blockers, process efficiency, team capacity
Questions they ask — Are sprints stable? Where are we blocked? Is work flowing smoothly? Are teams overloaded?
How they describe problems — Sprints feel chaotic. Work keeps getting sent back. Teams are constantly context-switching. We commit to things we don't deliver.
Metrics they care about most — Cycle Time, Reaction Time, Work-plan Inconsistency, Bulk Status Changes, Velocity, Sprint Predictability, Flow by Pace, Collaboration Intensity, Responsiveness
Engineering Manager
Team productivity, code quality, developer experience
Primary concerns — Code review speed, CI stability, rework reduction, collaboration health
Questions they ask — Are developers productive? Is code quality good? Are reviews happening fast? Are builds reliable?
How they describe problems — Developers are waiting too long for reviews. Builds keep failing. We're redoing work constantly. Not sure if people are collaborating or siloed.
Metrics they care about most — Tasks Moving Backwards, Collaboration Intensity, Responsiveness, Review Time, Declined Change Requests, Merge Rate, PR Throughput, Draft-to-Merge Lag, Review Depth, Reviewer Participation, CI Stability, Failure Rate, Rerun Rate, CI Duration, Untested Tasks, Quality Trend, Failure Hotspots, Planning vs Execution Split
Executive priorities
Every metric rolls up to one of nine executive priorities. Use this to connect a metric to the business case behind it.
Capital Efficiency
Are we getting ROI on engineering spend?
Turns engineering from cost center to managed investment
Cost by Initiative, Rework Cost, QA Failure Cost, Production Defect Cost, PMO Cost, Cost per Output
CFO
Competitive Velocity
Are we moving fast enough to win?
Faster release cycles = competitive advantage
Throughput, Lead Time, Cycle Time, Velocity, Review Time, PR Throughput, CI Duration, Deployment Frequency, Lead Time to Production
CTO
Strategic Execution
Are we building the right things?
Ensures work ties directly to business outcomes
Work Mix, Work-plan Inconsistency, Multiple Point Changes, Sprint Predictability, Strategy Alignment, Initiative Slippage Cost
CTO
Risk Management
What risks are forming right now?
Early warning system for delivery & quality failures
Priority Aging, Stale Tasks, Tasks Moving Backwards, Scope Creep, Strategic Scope Creep, PR Size/Risk, Change Hotspots, Blast Radius, Contributor Concentration, CI Stability, Failure Rate, Change Failure Rate, MTTR, Escaped Defect Rate, Untested Tasks, Quality Trend, Failure Hotspots, Production Defect Cost
CTO
Org Health & Talent
Are teams set up to succeed?
Prevents burnout, improves retention & performance
Bulk Status Changes, Collaboration Intensity, Responsiveness, Declined Change Requests, Review Depth, Reviewer Participation
VP Engineering
Board Confidence
Can we prove execution with data?
Builds credibility with board & investors
Velocity, Sprint Predictability, Strategy Alignment, Deployment Frequency, Initiative Slippage Cost
CTO
AI Impact
Is AI spend achieving performance change?
Validates AI investment ROI
Agent Adoption, Agent Session Volume, Time Spent with Agents, Prompt-to-Output Efficiency, AI Code Contribution %, AI Footprint in Delivery
CTO
AI Governance
How do we govern and control AI?
Controls AI spending and risk
Agent Usage by Platform, Model Usage Mix, Cost per Turn, Agent Tool Usage, Agent Failure/Error Rates, AI Spend by Task/Initiative, Planning vs Execution Split
CTO
Vendor Optimization
Right vendors on right work?
Optimizes external spend
Vendor Performance
CFO
Metric reference
All 65 metrics, grouped by where they live in the SDLC. Each entry includes the business question it answers, what a bad signal looks like, the actions to take, and who owns the response.
Planning & Flow
How work moves from request to done — throughput, cycle time, scope stability, sprint reliability, and strategic alignment.
Throughput
Number of work items completed per time period
Answers — "How many tasks are we completing and is it increasing?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead, Engineering Manager)
Data required — Task core record: task id, status, created date, completed date
Bad signal — Throughput declining 20%+ over 3 sprints → Team capacity is dropping or work complexity is increasing
What to do — (1) Review sprint retrospectives to identify blockers; (2) Analyze where time is going: meetings, rework, context switching; (3) Implement WIP limits or adjust team structure (owned by VP Engineering)
Common causes — Too many meetings, unclear requirements, technical debt slowing work, team attrition
Work Mix
Distribution of work across types (features, bugs, tech debt)
Answers — "Are we balancing new features vs. tech debt vs. bugs?"
SDLC area — PM · Primary persona — VP Engineering (also CPO/Product, CTO)
Data required — Task core record: type, status, labels/tags
Bad signal — >30% tech debt or >20% bug fixes → Not enough new features being built or quality is suffering
What to do — (1) Prioritize strategic work in next sprint; (2) Review why tech debt or bugs are accumulating; (3) Create explicit capacity allocation policy (70% features, 20% tech debt, 10% bugs) (owned by VP Engineering)
Common causes — Legacy code accumulation, pressure to ship fast without testing, no architectural planning
Priority Aging
Time high-priority items spend in backlog before starting
Answers — "Are critical items sitting too long unaddressed?"
SDLC area — PM · Primary persona — CTO (also VP Engineering, Delivery Lead)
Data required — Task core record: priority, created date, started date
Bad signal — Critical items sitting >2 weeks unstarted → High-value work is being blocked or deprioritized
What to do — (1) Immediately assign critical items to team leads; (2) Identify why priorities aren't being picked up: capacity, skills, blockers; (3) Establish priority escalation process and capacity reserves (owned by CTO)
Common causes — Teams overcommitted, unclear ownership, missing skills/context, dependencies
Lead Time
Total time from idea creation to production delivery
Answers — "How long does it take us to go from concept to customer value?"
SDLC area — PM · Primary persona — CTO (also CPO/Product, VP Engineering)
Data required — Task history: task id, created at, completed at, all state transitions
Bad signal — Lead time >30 days and increasing → Delivery pipeline has bottlenecks or too much bureaucracy
What to do — (1) Audit stages with longest wait times; (2) Map process flow and identify handoff delays; (3) Streamline approval gates, reduce batch sizes, increase deployment frequency (owned by CTO)
Common causes — Too many approval layers, large batch sizes, infrequent releases, manual processes
Cycle Time
Time from work start to completion (in-progress to done)
Answers — "Once we start work, how fast do we finish it?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead, Engineering Manager)
Data required — Task history: task id, status changes from in-progress to done
Bad signal — Cycle time >10 days or high variance → Work is too large, unclear, or teams are blocked
What to do — (1) Break down large tasks into smaller chunks; (2) Analyze where tasks spend most time: dev, review, testing; (3) Standardize task sizing, improve requirement clarity, optimize review process (owned by VP Engineering)
Common causes — Tasks too big, vague requirements, review bottlenecks, test environment delays
Reaction Time
Time between task creation and first status change
Answers — "How quickly do teams respond to new work requests?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Task history: task id, created at, first status change timestamp
Bad signal — Reaction time >2 days → Work is sitting idle or teams aren't engaged
What to do — (1) Flag stale work in daily standups; (2) Understand why work isn't being picked up: confusion, capacity, priorities; (3) Improve backlog grooming and make work ready before pulling (owned by Delivery Lead)
Common causes — Unclear acceptance criteria, team doesn't understand work, too much WIP, competing priorities
Stale Tasks
Count of tasks with no updates beyond threshold
Answers — "What work has gone dormant and might be abandoned?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Task history: task id, last changed at, status
Bad signal — >10% of backlog untouched for 30+ days → Backlog has dead weight or teams lost interest
What to do — (1) Archive or close stale tasks; (2) Determine which work is still valuable vs abandoned; (3) Implement regular backlog pruning sessions (owned by Delivery Lead)
Common causes — Scope changed, priority shifted, requirements unclear, original requestor left
Tasks Moving Backwards
Count of tasks regressing to earlier workflow states
Answers — "How often is work being sent back for rework?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead, Engineering Manager)
Data required — Task history: task id, status changes showing backward movement
Bad signal — >10% of tasks regress to earlier states → Quality issues or unclear requirements causing rework
What to do — (1) Immediately review failed tasks with teams; (2) Identify why work is being rejected: testing gaps, unclear specs, missed dependencies; (3) Implement definition of done checklists and better acceptance criteria (owned by VP Engineering)
Common causes — Incomplete requirements, inadequate testing, poor communication with stakeholders
Scope Creep
Changes to task scope after work begins
Answers — "How often do requirements change after we start building?"
SDLC area — PM · Primary persona — CTO (also CPO/Product, VP Engineering)
Data required — Task history: task id, field changes to points, description, acceptance criteria mid-flight
Bad signal — >15% of tasks have mid-flight scope changes → Requirements are unstable or stakeholders keep changing minds
What to do — (1) Lock scope on in-flight work immediately; (2) Review why requirements are changing: missing details, new info, stakeholder indecision; (3) Enforce change control process and better upfront scoping (owned by CTO)
Common causes — Rushed planning, unclear product vision, stakeholder misalignment, discovery during build
Work-plan Inconsistency
Mismatch between planned vs actual work
Answers — "Are we sticking to our plans or constantly adjusting?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Task history + sprints: tasks added/removed mid-sprint, commitment vs delivery
Bad signal — <60% sprint commitment met → Teams can't predict or scope isn't stable
What to do — (1) Review what's causing plan changes in retro; (2) Analyze commitment patterns vs delivery patterns; (3) Improve estimation, reduce mid-sprint additions, stabilize capacity (owned by VP Engineering)
Common causes — Over-optimistic estimates, surprise work, unclear priorities, interruptions
Bulk Status Changes
Multiple tasks updated simultaneously (automation or process issues)
Answers — "Are we gaming metrics or batching work unnaturally?"
SDLC area — PM · Primary persona — Delivery Lead (also VP Engineering)
Data required — Task history: timestamps of status changes, changed by user
Bad signal — Multiple identical status updates at once → Process gaming or manual workarounds
What to do — (1) Audit who made bulk changes and why; (2) Determine if it's legitimate process vs metric manipulation; (3) Automate status updates or fix tool configurations (owned by Delivery Lead)
Common causes — Tool migrations, process shortcuts, metric gaming, manual cleanup of stale work
Multiple Point Changes
Tasks with story points changed multiple times
Answers — "Are estimates unstable or requirements unclear?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Task history: point value changes, timestamps
Bad signal — >20% of tasks have point changes → Estimates are unstable or requirements unclear
What to do — (1) Review tasks with frequent changes; (2) Understand why estimates are being revised: complexity discovery, scope changes; (3) Improve estimation practices and requirement clarity (owned by VP Engineering)
Common causes — Poor initial estimates, scope creep, technical unknowns, pressure to commit early
Velocity
Story points completed per sprint
Answers — "Is our delivery capacity stable and predictable?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Sprints + task history: sprint id, points completed per sprint
Bad signal — Velocity variance >30% sprint-to-sprint → Capacity is unpredictable or work sizing inconsistent
What to do — (1) Normalize for holidays/PTO in sprint planning; (2) Identify root cause: team changes, work complexity, external dependencies; (3) Stabilize team composition and work sizing practices (owned by VP Engineering)
Common causes — Team turnover, vacation spikes, work mix shifts, external dependencies
Sprint Predictability
Ratio of committed vs delivered work in sprints
Answers — "Do we deliver what we commit to in sprints?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Sprints + task history: planned vs actual completion
Bad signal — <70% of committed work delivered → Over-committing or too many disruptions
What to do — (1) Reduce commitments in next sprint; (2) Track why work didn't complete: blockers, interrupts, estimates; (3) Implement buffer capacity (80% commitment), reduce mid-sprint additions (owned by Delivery Lead)
Common causes — Optimistic planning, unplanned work, unclear requirements, external dependencies
Flow by Pace
Distribution of cycle times across different work cadences
Answers — "Does our pace vary by sprint duration or team cadence?"
SDLC area — PM · Primary persona — VP Engineering (also Delivery Lead)
Data required — Sprints + task history: sprint duration, cycle times by sprint
Bad signal — Cycle times vary 2x+ by sprint length → Cadence mismatch or batch sizing issues
What to do — (1) Experiment with different sprint lengths; (2) Analyze if shorter/longer sprints improve flow; (3) Align sprint length to team's natural delivery rhythm (owned by Delivery Lead)
Common causes — Work doesn't fit sprint boundaries, review overhead too high, batch size mismatch
Collaboration Intensity
Comment and interaction volume on tasks
Answers — "Are teams actively collaborating or working in silos?"
SDLC area — PM · Primary persona — Engineering Manager (also Delivery Lead)
Data required — Task comments: comment count per task, unique participants
Bad signal — Comments per task <2 or >20 → Either no communication or excessive thrashing
What to do — (1) Review tasks with extreme comment counts; (2) Understand if silence means clarity or neglect; noise means confusion; (3) Clarify when to collaborate (complex work) vs heads-down (clear work) (owned by Engineering Manager)
Common causes — Tasks too simple/clear for collaboration, or too vague requiring constant clarification
Responsiveness
Time between comments or updates on tasks
Answers — "How quickly do teams communicate and unblock each other?"
SDLC area — PM · Primary persona — Engineering Manager (also Delivery Lead)
Data required — Task comments: comment timestamps, response lag
Bad signal — Comment response time >24 hours → Teams are siloed or too busy to collaborate
What to do — (1) Flag delayed responses in standups; (2) Identify if it's lack of time, unclear ownership, or disconnected teams; (3) Establish response time SLAs, improve notification settings (owned by Engineering Manager)
Common causes — Time zone differences, too much WIP, unclear ownership, notification overload
Strategy Alignment
Percentage of work mapped to strategic initiatives
Answers — "Are we spending time on work that matters to the business?"
SDLC area — PM/Strategy · Primary persona — CTO (also CPO/Product, CFO)
Data required — Hierarchy mapping: tasks to epics to initiatives to goals
Bad signal — <50% of work mapped to strategic goals → Lots of reactive work or unclear priorities
What to do — (1) Pause new work not tied to strategy; (2) Audit what teams are working on and why; (3) Create clear strategic priorities and capacity allocation rules (owned by CTO)
Common causes — No clear strategy, too much maintenance/tech debt, customer escalations, unclear priorities
Cost by Initiative
Engineering spend allocated to strategic initiatives
Answers — "Where is our engineering budget actually going?"
SDLC area — PM/Strategy · Primary persona — CFO (also CTO, CPO/Product)
Data required — Hierarchy mapping + task effort: initiative → tasks → time spent
Bad signal — High-cost initiatives with low value → Money being spent on wrong things
What to do — (1) Review initiative ROI and consider pausing low-value work; (2) Assess if cost is due to complexity, inefficiency, or scope bloat; (3) Reallocate budget to high-impact initiatives (owned by CFO)
Common causes — Scope creep, inefficient execution, wrong team assignment, unclear requirements
Strategic Scope Creep
New work added to initiatives after start
Answers — "Are initiatives growing beyond original scope?"
SDLC area — PM/Strategy · Primary persona — CTO (also CPO/Product, VP Engineering)
Data required — Hierarchy mapping + task history: tasks added post-initiative start
Bad signal — >30% new tasks added post-initiative start → Initiatives poorly defined or requirements changing
What to do — (1) Freeze scope on active initiatives; (2) Determine if additions are necessary vs nice-to-haves; (3) Improve initiative planning and change control (owned by CTO)
Common causes — Incomplete scoping, stakeholder changes, competitive pressure, technical unknowns
Code & Review
Pull-request flow and code-review health — review speed, PR size and risk, reviewer load, and code ownership.
Review Time
Time from PR open to merge
Answers — "How long does code sit waiting for review?"
SDLC area — SCM · Primary persona — VP Engineering (also Engineering Manager)
Data required — Pull requests: created, merged timestamps
Bad signal — PR review time >24 hours median → Review bottlenecks slowing delivery
What to do — (1) Flag old PRs in team chat daily; (2) Identify if reviewers are overloaded or PRs are too complex; (3) Distribute review load, set review SLAs, pair programming (owned by VP Engineering)
Common causes — Too few reviewers, PRs too large, reviewers busy, unclear review expectations
Declined Change Requests
PRs closed without merging
Answers — "How much work is being abandoned or rejected?"
SDLC area — SCM · Primary persona — VP Engineering (also Engineering Manager)
Data required — Pull requests: state, closed but not merged
Bad signal — >15% PRs closed without merge → Work being abandoned or quality bar too low
What to do — (1) Review reasons for rejected PRs; (2) Understand if rejections are due to quality, priority changes, or misalignment; (3) Improve upfront planning and code quality standards (owned by VP Engineering)
Common causes — Speculative work, misaligned with goals, quality issues, priority shifts
Merge Rate
Percentage of PRs successfully merged
Answers — "Are we shipping most of what we build?"
SDLC area — SCM · Primary persona — VP Engineering (also Engineering Manager)
Data required — Pull requests: state (merged vs closed)
Bad signal — <80% merge rate → Too much abandoned work or quality problems
What to do — (1) Review rejected PRs for patterns; (2) Identify if rejections are good (quality bar) or bad (wasted work); (3) Improve work intake process and definition of done (owned by VP Engineering)
Common causes — Experimental work not panning out, poor code quality, changing priorities
PR Throughput
Number of PRs merged per time period
Answers — "How much code are we shipping?"
SDLC area — SCM · Primary persona — VP Engineering (also Engineering Manager)
Data required — Pull requests: merged timestamp
Bad signal — PR merges declining >20% → Team slowing down or PRs getting larger
What to do — (1) Review team capacity and recent changes; (2) Analyze if PR size is increasing or review is slowing; (3) Optimize PR sizing and review process (owned by VP Engineering)
Common causes — Team capacity down, PRs too large, review bottlenecks, CI instability
PR Size/Risk
Lines changed, files modified per PR
Answers — "Are PRs appropriately sized or risky large changes?"
SDLC area — SCM · Primary persona — VP Engineering (also Engineering Manager)
Data required — Pull requests: additions, deletions, changed files count
Bad signal — PRs >500 lines or >20 files median → Changes too large to review effectively
What to do — (1) Split large PRs into smaller chunks; (2) Understand why PRs are big: work not broken down, refactoring, migrations; (3) Establish PR size limits and train on incremental delivery (owned by VP Engineering)
Common causes — Work not decomposed, architectural changes, lack of guidance on PR sizing
Draft-to-Merge Lag
Time PRs spend in draft state before ready for review
Answers — "Is work sitting in progress too long before review?"
SDLC area — SCM · Primary persona — Engineering Manager (also VP Engineering)
Data required — Pull requests: created, draft flag removed timestamp, merged
Bad signal — Draft time >5 days median → Work sitting incomplete too long
What to do — (1) Review stalled drafts with authors; (2) Identify blockers: waiting for decisions, complex problem, external dependency; (3) Improve work breakdown and unblock dependencies earlier (owned by Engineering Manager)
Common causes — Work too complex, waiting on decisions, external dependencies, unclear requirements
Change Hotspots
Files or areas with high modification frequency
Answers — "What parts of the codebase are changing the most?"
SDLC area — SCM · Primary persona — CTO (also VP Engineering)
Data required — PR files: file path, change frequency
Bad signal — Same files changed >50 times per quarter → Code instability or poor architecture
What to do — (1) Audit hotspot files for quality issues; (2) Determine if changes are features, bugs, or refactoring; (3) Refactor hotspots or establish ownership for stability (owned by CTO)
Common causes — Legacy code, architectural debt, feature area churn, unclear ownership
Blast Radius
Number of files/systems impacted by changes
Answers — "How broad is the impact of each change?"
SDLC area — SCM · Primary persona — CTO (also VP Engineering)
Data required — PR files: file paths, dependency analysis
Bad signal — Changes touching >50 files median → High coupling or risky large changes
What to do — (1) Require extra review for large-impact PRs; (2) Assess if coupling is necessary or technical debt; (3) Improve modularity and reduce cross-cutting concerns (owned by CTO)
Common causes — Monolithic architecture, tight coupling, global changes, migrations
Review Depth
Comment volume and reviewer engagement on PRs
Answers — "Are we thoroughly reviewing code or rubber-stamping?"
SDLC area — SCM · Primary persona — Engineering Manager (also VP Engineering)
Data required — PR comments/reviews: comment count, unique reviewers
Bad signal — Comments per PR <1 median → Rubber-stamping reviews or trivial changes
What to do — (1) Audit a sample of PRs for review quality; (2) Determine if lack of comments is good (clean code) or bad (not reviewing); (3) Train on effective code review and establish review standards (owned by Engineering Manager)
Common causes — Trivial changes, time pressure, unclear review expectations, trust vs thoroughness
Reviewer Participation
Distribution of review workload across team
Answers — "Is review burden shared fairly or concentrated?"
SDLC area — SCM · Primary persona — Engineering Manager (also VP Engineering)
Data required — PR comments/reviews: reviewer ids, review counts
Bad signal — 80% of reviews by 20% of reviewers → Review load concentrated, bottleneck risk
What to do — (1) Distribute reviews more evenly across team; (2) Identify why certain people review more: expertise, availability, others opting out; (3) Balance review workload and train more reviewers (owned by Engineering Manager)
Common causes — Expertise concentration, only seniors review, lack of rotation, uneven workload
Contributor Concentration
Bus factor - how many developers touch critical code
Answers — "What happens if key developers leave?"
SDLC area — SCM · Primary persona — CTO (also VP Engineering)
Data required — Contributors + PR files: contributor IDs, file paths
Bad signal — Bus factor <3 for critical systems → Key person risk in important areas
What to do — (1) Document critical systems immediately; (2) Map who knows what and where knowledge gaps exist; (3) Pair programming, documentation, cross-training (owned by CTO)
Common causes — Expertise silos, legacy systems, no documentation, complex domains
CI / CD
Build pipeline reliability and speed.
CI Stability
Pipeline success rate
Answers — "Are our builds reliable or constantly breaking?"
SDLC area — CI/CD · Primary persona — VP Engineering (also Engineering Manager)
Data required — Workflow runs: status (success vs failure)
Bad signal — Build success rate <85% → Builds unreliable, slowing teams down
What to do — (1) Identify most common build failures; (2) Determine if failures are tests, infrastructure, or code quality; (3) Fix flaky tests, improve infrastructure, increase test quality (owned by VP Engineering)
Common causes — Flaky tests, infrastructure issues, test environment instability, timing issues
Failure Rate
Percentage of failed pipeline runs
Answers — "How often do builds fail?"
SDLC area — CI/CD · Primary persona — VP Engineering (also Engineering Manager)
Data required — Workflow runs: status
Bad signal — Build failure rate >20% → Too many broken builds
What to do — (1) Review recent failures for patterns; (2) Identify if failures are preventable or inherent complexity; (3) Improve pre-commit checks, better test practices (owned by VP Engineering)
Common causes — Inadequate local testing, flaky tests, brittle tests, environmental issues
Rerun Rate
Pipelines requiring multiple attempts to pass
Answers — "Are flaky tests or infrastructure wasting time?"
SDLC area — CI/CD · Primary persona — VP Engineering (also Engineering Manager)
Data required — Workflow runs: attempts count
Bad signal — Builds requiring >1 attempt >30% → Flaky infrastructure or tests
What to do — (1) Prioritize fixing flaky tests; (2) Audit which tests or stages are flaky; (3) Stabilize test infrastructure, remove non-deterministic tests (owned by VP Engineering)
Common causes — Flaky tests, timing issues, infrastructure instability, environmental dependencies
CI Duration
Time pipelines take to complete
Answers — "How long do developers wait for builds?"
SDLC area — CI/CD · Primary persona — VP Engineering (also Engineering Manager)
Data required — Workflow runs: start, end timestamps
Bad signal — Build time >20 minutes median → Slow feedback loop for developers
What to do — (1) Profile build to find slowest stages; (2) Determine if slowness is tests, compilation, or deployment; (3) Parallelize, cache, optimize slow stages (owned by VP Engineering)
Common causes — Monolithic builds, slow tests, no caching, sequential execution
CI Bottlenecks
Stages or jobs causing delays
Answers — "Where in our pipeline are we losing time?"
SDLC area — CI/CD · Primary persona — VP Engineering (also Engineering Manager)
Data required — Workflow runs: job durations, dependencies
Bad signal — One stage taking >50% of build time → Specific bottleneck slowing everything
What to do — (1) Optimize the slowest stage first; (2) Profile why stage is slow: computation, I/O, waiting; (3) Parallelize, cache, or optimize bottleneck stage (owned by VP Engineering)
Common causes — Sequential dependencies, slow tests, inefficient scripts, resource limits
Delivery, Reliability & Quality
Getting changes to production safely — deployment frequency, failure rates, incident recovery, and test quality.
Deployment Frequency
How often code reaches production
Answers — "How often are we shipping to customers?"
SDLC area — Delivery · Primary persona — CTO (also VP Engineering)
Data required — Deployments: environment, deployment timestamp
Bad signal — Deploying <1x per week → Release cadence too slow
What to do — (1) Reduce batch size of releases; (2) Identify deployment blockers: manual processes, fear, long cycles; (3) Automate deployment, increase confidence with testing (owned by CTO)
Common causes — Manual deployments, lack of confidence, large batches, infrequent release windows
Lead Time to Production
Time from commit to production deployment
Answers — "How fast can we get changes to customers?"
SDLC area — Delivery · Primary persona — CTO (also VP Engineering)
Data required — Deployments + commits: commit timestamp, deployment timestamp
Bad signal — Commit to prod >7 days median → Deployment pipeline has delays
What to do — (1) Map deployment stages and find delays; (2) Identify manual gates, approvals, or testing delays; (3) Automate gates, reduce batch size, continuous deployment (owned by CTO)
Common causes — Manual approvals, large batches, testing delays, deployment windows
Change Failure Rate
Percentage of deployments causing incidents
Answers — "How often do our releases cause problems?"
SDLC area — Delivery · Primary persona — CTO (also VP Engineering)
Data required — Deployments + incidents: deployment linked to incident
Bad signal — >15% deployments cause incidents → Quality gates not catching issues
What to do — (1) Increase scrutiny on next release; (2) Audit what types of issues escape: config, code, data; (3) Improve testing, monitoring, rollback procedures (owned by CTO)
Common causes — Inadequate testing, environmental differences, configuration errors, monitoring gaps
MTTR
Mean time to resolve incidents
Answers — "How quickly do we fix production issues?"
SDLC area — Reliability · Primary persona — CTO (also VP Engineering)
Data required — Incidents: opened timestamp, resolved timestamp
Bad signal — Mean time to resolve >4 hours → Incidents taking too long to fix
What to do — (1) Review recent incidents for patterns; (2) Identify if delays are detection, diagnosis, or fix; (3) Improve monitoring, runbooks, rollback procedures (owned by CTO)
Common causes — Poor monitoring, unclear runbooks, complex systems, slow rollback
Escaped Defect Rate
Bugs reaching production that should have been caught
Answers — "How effective is our testing?"
SDLC area — Reliability · Primary persona — CTO (also VP Engineering, Engineering Manager)
Data required — Incidents: severity, linked task/deployment
Bad signal — >5% defects escaping to production → Testing not effective
What to do — (1) Review what defects escaped and why; (2) Analyze testing gaps: no tests, wrong tests, environmental issues; (3) Expand test coverage, improve test quality, better environments (owned by CTO)
Common causes — Inadequate test coverage, poor test quality, env differences, time pressure
Untested Tasks
Work shipped without automated test coverage
Answers — "Are we skipping quality gates?"
SDLC area — Quality · Primary persona — VP Engineering (also Engineering Manager)
Data required — Test results + tasks: tasks with no test execution
Bad signal — >20% tasks ship without tests → Skipping quality gates
What to do — (1) Require tests for all production code; (2) Understand why tests are skipped: time pressure, legacy, unclear standards; (3) Enforce testing standards and allocate time for testing (owned by VP Engineering)
Common causes — Time pressure, unclear standards, legacy code, lack of test infrastructure
Quality Trend
Test failure rate over time
Answers — "Is quality improving or degrading?"
SDLC area — Quality · Primary persona — VP Engineering (also Engineering Manager)
Data required — Test results: status, timestamp
Bad signal — Test failure rate increasing → Quality degrading over time
What to do — (1) Fix failing tests immediately; (2) Determine if new tests are flaky or code quality is declining; (3) Improve test quality and code review standards (owned by VP Engineering)
Common causes — Rushed code, declining standards, flaky tests, technical debt
Failure Hotspots
Tests or areas with high failure rates
Answers — "Where are our persistent quality problems?"
SDLC area — Quality · Primary persona — Engineering Manager (also VP Engineering)
Data required — Test results: suite/check id, failure count
Bad signal — Same tests failing repeatedly → Persistent quality problems
What to do — (1) Prioritize fixing chronic failures; (2) Identify root cause: flaky tests, code quality, design issues; (3) Refactor problem areas or remove flaky tests (owned by Engineering Manager)
Common causes — Flaky tests, poorly designed code, architectural issues, environmental dependencies
Release Readiness
Percentage of work passing quality gates
Answers — "Are we ready to ship on schedule?"
SDLC area — Quality · Primary persona — VP Engineering (also Delivery Lead)
Data required — Test results + tasks: tasks passing all checks
Bad signal — <80% of work passing gates → Not ready to ship on schedule
What to do — (1) Pause new work and fix failing work; (2) Identify why work is failing: quality, incomplete, dependencies; (3) Improve definition of done and earlier testing (owned by VP Engineering)
Common causes — Rushed work, unclear standards, late testing, missing requirements