Growth Engineering: Experiments, Analytics and Referrals

Growth engineering is software engineering applied to acquisition, activation and retention: building the experimentation platform, event pipeline, onboarding flows, referral mechanics and technical SEO that let a team test growth ideas quickly and trust the results. Its core is three systems: deterministic feature flags for experiments, a typed event tracking plan with identity stitching, and analysis that respects sample size and guardrail metrics. Onboarding changes, lifecycle emails and referral programs are experiments that run on top of those three.

Key takeaways

• Build measurement before tactics. An experiment you cannot measure correctly is worse than no experiment, because it produces confident wrong answers.
• Assign variants deterministically by hashing the user ID with the experiment key, and log an exposure only when the user actually sees the variant.
• Calculate the sample size before starting and write down the primary metric and guardrails. Peeking at results and stopping early inflates false positives.
• Keep a tracking plan with typed events, send business-critical events from the server, and stitch anonymous and signed-in identities at signup.
• Activation is usually the highest-leverage step in the funnel. Define it as an early action that predicts retention, then make it easier to reach.
• Page speed, mobile layouts and crawlable pages are growth infrastructure, not marketing chores.

What growth engineering owns

Growth hacking is a collection of tactics: a launch, a clever integration, a viral post. Growth engineering builds the systems that make growth repeatable and measurable, so that a team can run many small experiments a month and know which ones worked.

Feature flags and experiment assignment

Feature flags decouple deploying code from releasing it. For experiments they also have to assign users to variants in a way that is stable, unbiased and cheap to evaluate on every request.

import { createHash } from "node:crypto";
 
// Stable bucket in the range 0 to 9999 for a user and a salt
function bucket(userId: string, salt: string): number {
  const digest = createHash("sha256").update(`${salt}:${userId}`).digest();
  return digest.readUInt32BE(0) % 10000;
}
 
interface Experiment {
  key: string;
  allocation: number; // share of eligible users enrolled, 0 to 1
  variants: { name: string; weight: number }[]; // weights sum to 1
}
 
export function assign(exp: Experiment, userId: string): string | null {
  if (bucket(userId, exp.key) >= exp.allocation * 10000) return null; // not enrolled
  const point = bucket(userId, `${exp.key}:variant`) / 10000; // independent second hash
  let cumulative = 0;
  for (const v of exp.variants) {
    cumulative += v.weight;
    if (point < cumulative) return v.name;
  }
  return exp.variants[exp.variants.length - 1].name;
}

The design choices that matter:

• Deterministic assignment. The same user always lands in the same variant, on every server, with no lookup. Salting the hash with the experiment key keeps experiments independent of each other.
• Separate enrollment and variant hashes. Changing the allocation from 10% to 50% enrolls new users without moving anyone already enrolled into a different variant.
• Log exposure at render, not at assignment. Record an experiment_exposed event when the user actually sees the variant. Users assigned but never exposed dilute the effect toward zero.
• Evaluate on the server. Client-side evaluation flickers from control to variant and changes behavior by itself. Render the right variant on the first paint.
• Fail to control. If the flag service is unreachable, the SDK should fall back to cached configuration, then to the control experience. Never block page rendering on a flag fetch.

Do not build the whole platform yourself. PostHog, GrowthBook, LaunchDarkly and Unleash all provide flags with experimentation, and OpenFeature gives you a vendor-neutral SDK interface so the choice stays reversible. Build only the glue: your identity model, exposure events and the warehouse tables analysis reads from.

Experiment math you cannot skip

Sample size, before you start

To compare two conversion rates, the users needed per variant are approximately:

n per variant = (z_alpha + z_beta)^2 x (p1(1-p1) + p2(1-p2)) / (p2-p1)^2

With a two-sided significance level of 0.05, z_alpha is 1.96; with 80% power, z_beta is 0.84. Worked through for a signup page converting at 5%, where you want to detect a lift to 5.5%:

(1.96 + 0.84)^2                = 7.84
p1(1-p1) + p2(1-p2)            = 0.0475 + 0.051975 = 0.099475
(p2-p1)^2                      = 0.005^2 = 0.000025
n per variant = 7.84 x 0.099475 / 0.000025 = about 31,200

So roughly 62,400 users across both variants. Divide by eligible daily traffic to get the duration, then round up to whole weeks so every weekday is represented equally. If the answer is three months, test a bolder change with a bigger expected effect, or test further up the funnel where traffic is higher. You can check the arithmetic with a library:

from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
 
effect = proportion_effectsize(0.055, 0.05)  # Cohen's h for the two rates
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8, alternative="two-sided")
print(round(n))  # users per variant, close to the hand calculation

Rules that keep results honest

1. Write the primary metric and guardrails before launch. Choosing the metric after seeing the data is how teams find wins that are not there.
2. Do not stop when the p-value first dips below 0.05. Checking repeatedly and stopping at the first significant result inflates the false positive rate well beyond 5%. Run to the planned sample size, or use a sequential method designed for continuous monitoring, which many experimentation platforms offer.
3. Check for sample ratio mismatch first. If a 50/50 split produced visibly unequal groups, something in assignment, redirects, caching or bot filtering is broken, and the result cannot be trusted.
4. Watch guardrails. Page latency, error rate, refunds, unsubscribes and revenue per user. A variant that lifts signups and doubles refund requests did not win.
5. Segment new and existing users. Novelty inflates early engagement with anything new; existing users sometimes resist any change. Look at the trend over the test period, not only the total.
6. Reduce variance where you can. CUPED, which adjusts each user's outcome using their own pre-experiment behavior, can shorten tests on metrics with history.

A sample ratio check takes four lines:

from scipy.stats import chisquare
 
observed = [50410, 49120]                 # users per variant (example numbers)
stat, p = chisquare(observed, [sum(observed) / 2] * 2)
print("sample ratio mismatch" if p < 0.001 else "split looks healthy", p)

The event tracking plan

Analytics fails quietly. Events get renamed, fire twice, lose properties or stop firing after a refactor, and the dashboard keeps drawing lines. A tracking plan in the repository is the fix: every event defined once, typed, owned and validated.

events:
  signup_completed:
    description: Account created and email verified
    owner: growth
    source: server
    properties:
      method: { type: string, enum: [email, google, sso] }
      referral_code: { type: string, required: false }
  project_created:
    description: A project was created
    owner: core-product
    source: server
    properties:
      template_id: { type: string, required: false }
      is_first: { type: boolean }
  experiment_exposed:
    description: User saw an experiment variant
    owner: growth
    source: client
    properties:
      experiment_key: { type: string }
      variant: { type: string }

Practices that keep the data trustworthy:

• One naming convention. object_action in snake case, past tense, is easy to scan. Whatever you choose, enforce it.
• Generate typed tracking functions from the plan, so a misspelled event or missing property fails the build instead of the dashboard.
• Validate at ingestion. Quarantine events that do not match the schema rather than silently storing them.
• Attach context to every event: user ID, anonymous ID, organization ID, session ID, app version and current experiment assignments.
• Give every event a unique ID and deduplicate in the pipeline. Components that render twice will send events twice.
• Send business-critical events from the server. Ad blockers and browser privacy protections drop a share of client-side analytics. Signups, activations and payments should come from the backend, where they are complete.
• Stitch identities at signup. Before signup the user has an anonymous ID from a first-party cookie; at signup, link it to the user ID so the pre-signup journey (landing page, experiment exposure, referral) stays attached.
• Respect consent. Client-side tracking may need consent depending on where your users are. Build the consent state into the tracking layer, not around it.

From the collector, events usually go both to a product analytics tool and to the warehouse, where experiment analysis and revenue data can be joined. Our guide to the modern data engineering stack covers that pipeline.

Activation: define it with data, then shorten the path

Activation is the moment a new user first gets real value. It matters because every later stage, from retention to referrals to revenue, is drawn from the users who activated.

Choose the activation event empirically. List candidate early actions (created a project, invited a teammate, connected an integration, shared a result), then compare the retention of users who did each within their first week against those who did not. Pick an action that is strongly associated with retention, easy to explain and reachable early. Association is not causation, so confirm it with an experiment that makes the action easier and watch whether retention follows.

Weekly activation rate for a chosen event:

select date_trunc('week', u.created_at) as signup_week,
       count(*) as signups,
       avg(case when exists (
             select 1 from events e
             where e.user_id = u.id
               and e.name = 'project_shared'
               and e.occurred_at < u.created_at + interval '7 days'
           ) then 1.0 else 0.0 end) as activation_rate_7d
from users u
group by 1
order by 1;

Then shorten the path:

• Remove steps between signup and value. Every field, confirmation and empty screen is a place to leave.
• Start users with something. Templates, sample data or an import beat an empty state.
• Ask one or two questions at signup (role, main use case) and route people to the relevant first step.
• Trigger lifecycle messages from behavior. If the activation event has not fired within a day or two, send one specific next step with a link straight to it. Re-check the event right before sending, so users who already finished do not get a "finish setup" email.

For an early product, activation and weekly retention are often the only growth metrics worth watching. Our MVP development guide includes a cohort retention query that pairs with the one above.

Referral systems that do not get abused

A referral program needs codes, attribution, rewards and, from day one, fraud controls.

create table referral_codes (
  code        text primary key,
  user_id     text not null unique,
  created_at  timestamptz not null default now()
);
 
create table referrals (
  referred_user_id text primary key,       -- a user can be referred only once
  code             text not null references referral_codes (code),
  signed_up_at     timestamptz not null,
  qualified_at     timestamptz,            -- set when the referred user activates or pays
  reward_status    text not null default 'pending'
);

Rules that keep it healthy:

• Reward on qualification, not signup. Pay out when the referred user activates or makes a first payment. Rewards for bare signups attract fake accounts.
• Attribute with a window. Store the code on the first visit in a first-party cookie, carry it through signup, and expire it after a defined period.
• Block self-referral and farming. Check for the same payment fingerprint, matching devices or network patterns, disposable email domains, and unusual signup velocity per code. Hold large rewards for manual review.
• Make sharing effortless. A copyable link, prefilled messages for the channels your users already use, and an invite flow inside the product at the moment users get value.

Measure it with the viral coefficient: k = i x c, where i is invitations sent per user and c is the share of invitations that convert. A k above 1 means each user brings in more than one new user, which few products sustain. Below 1, referrals still lower acquisition cost, and the cycle time, how long it takes a new user to invite the next one, matters as much as k.

Performance and technical SEO as growth infrastructure

Landing pages, pricing pages and signup flows are where speed turns into conversion. Google's Core Web Vitals are a reasonable shared target: Largest Contentful Paint at or under 2.5 seconds, Interaction to Next Paint at or under 200 milliseconds and Cumulative Layout Shift at or under 0.1, measured at the 75th percentile of real page loads. Interaction to Next Paint replaced First Input Delay as a Core Web Vital in 2024. Use field data from real users, not just a lab score from your own laptop. Our guide to Next.js performance optimization covers the fixes in detail.

Mobile layouts deserve their own check. A signup form whose submit button is pushed off-screen at phone widths loses conversions silently, because nobody on the team browses at that width. wontfit runs in CI, renders pages at phone widths and fails the build when an element overflows, naming the element.

Technical SEO checklist for pages that should rank:

• Content rendered on the server or generated statically, so crawlers see it without running JavaScript
• One canonical URL per page, and redirects for the variants
• An XML sitemap with accurate last-modified dates, and a robots.txt that does not block CSS or JavaScript
• Structured data that matches visible content (Organization, Article, Product, BreadcrumbList)
• Internal links from established pages to new ones
• Unique titles and descriptions written for the query, not the brand

Two outdated tactics to drop. Google retired HowTo rich results in 2023, limited FAQ rich results to well-known government and health sites the same year, and stopped showing FAQ rich results altogether in May 2026, so adding that markup will not change how your pages look in Google Search. And programmatic pages generated at scale without unique, useful content fall under Google's spam policy on scaled content abuse. Generate pages from real data that differs page by page, or do not generate them.

What goes wrong in production

The flag service goes down. If the SDK blocks rendering while it waits for configuration, a flag vendor outage becomes your outage. Cache configuration locally, fall back to control and set short timeouts.

Flag debt. Experiments end, flags stay, and dead code paths keep running. Give every flag an owner and an expiry date, and report expired flags weekly.

Cached variants. A CDN caches the HTML for variant B and serves it to everyone, or to the wrong bucket. Assign at the edge, vary the cache by bucket, or keep experiment pages uncached.

Double-fired events. A component renders twice and sends signup_completed twice, inflating conversion. Unique event IDs and pipeline deduplication fix it; tracking critical events on the server avoids it.

Exposure logged at assignment. Everyone assigned is counted, including users who never reached the changed page. The measured effect shrinks toward zero and real wins look like noise.

Overlapping experiments. Two tests change the same page and interact. Use mutually exclusive layers for experiments on the same surface.

Day boundaries in different time zones. The dashboard uses UTC, the warehouse uses local time, and daily numbers disagree. Pick one time zone for metrics and document it.

Referral fraud on launch day. A generous reward announced publicly attracts farming almost immediately. Launch with qualification rules, velocity limits and manual review already in place.

Frequently Asked Questions

What does a growth engineer do?

A growth engineer builds the systems that let a company test and scale ways of acquiring, activating and retaining users. That includes feature flags and experimentation infrastructure, event tracking and analytics pipelines, onboarding flows, lifecycle messaging triggers, referral systems and technical SEO. The role combines product engineering with enough statistics to judge whether an experiment really worked.

How long should an A/B test run?

Run an A/B test until it reaches the sample size you calculated before starting, based on your baseline conversion rate and the smallest effect you care about. Round the duration up to whole weeks so that every day of the week is equally represented. Stopping as soon as results look significant inflates false positives unless you use a sequential testing method designed for it.

What is a good K-factor?

A K-factor above 1 means each user brings in more than one new user, which produces viral growth but is rare and hard to sustain. Most products have a K-factor well below 1, where referrals reduce acquisition cost rather than drive growth on their own. How quickly referred users invite others, the viral cycle time, matters as much as the coefficient.

Should you track analytics events on the client or the server?

Track business-critical events such as signups, activations and payments on the server, because ad blockers and browser privacy protections drop a share of client-side events. Use client-side tracking for interface interactions and experiment exposure that only the browser can see. Link the two with a shared anonymous ID that is stitched to the user ID at signup.

How do you choose an activation metric?

List the early actions new users can take, then compare the retention of users who completed each one in their first week with those who did not. Choose an action that is strongly associated with retention, easy to explain and reachable early in the user's life. Then confirm the relationship with an experiment that makes the action easier to reach.

Need help with growth engineering?

We are senior engineers who build systems and then run them: if we built it, we are on the pager. For growth work that means experiment assignment that stays correct under caching, event pipelines you can trust and pages that stay fast. We can build the foundation or audit the one you have.

• Software development
• Startup tech consulting
• Talk to an engineer