1.3Feature Flag Testing Strategy

Quality-Gated Rollout Pattern

# Feature-flagged AI summarization with quality gates and fallback
import ldclient
from ldclient.config import Config
from ldclient import Context

ldclient.set_config(Config("sdk-key-production"))
client = ldclient.get()

def get_ai_summary(document, user_context):
    """Feature-flagged AI summarization with automatic quality gate."""
    context = Context.builder(user_context["user_id"]) \
        .set("plan", user_context["plan"]) \
        .set("region", user_context["region"]) \
        .build()

    # Check if this user should get the new AI summary feature
    if client.variation("ai-summary-v2", context, False):
        try:
            summary = call_new_ai_summary_endpoint(document)

            # Quality gate: verify the summary meets minimum quality bar
            quality_score = evaluate_summary_quality(summary, document)
            if quality_score < 0.7:
                # Track degraded quality as a metric
                client.track("ai-summary-quality-degraded", context,
                           metric_value=quality_score)
                # Fall back to old implementation
                return call_legacy_summary_endpoint(document)

            client.track("ai-summary-v2-success", context,
                       metric_value=quality_score)
            return summary

        except Exception as e:
            client.track("ai-summary-v2-error", context)
            return call_legacy_summary_endpoint(document)
    else:
        return call_legacy_summary_endpoint(document)

Progressive Rollout Plan

A disciplined rollout progresses through gates, each with specific quality signals:

# progressive-rollout-plan.yaml
feature: ai-summary-v2
rollout_stages:
  - name: internal_dogfood
    percentage: 0%
    targeting: "email ends with @ourcompany.com"
    duration: 3 days
    quality_gates:
      - error_rate < 1%
      - p95_latency < 3s
      - quality_score_avg > 0.8
    rollback_trigger: "any gate fails for 15 minutes"

  - name: beta_users
    percentage: 5%
    targeting: "plan == 'beta'"
    duration: 5 days
    quality_gates:
      - error_rate < 0.5%
      - p95_latency < 2.5s
      - quality_score_avg > 0.82
      - user_satisfaction_score > 4.0
    rollback_trigger: "any gate fails for 30 minutes"

  - name: gradual_rollout
    percentage_stages: [10%, 25%, 50%, 75%, 100%]
    advance_interval: 24 hours
    quality_gates:
      - error_rate < 0.3%
      - p95_latency < 2s
      - quality_score_avg > 0.85
      - no_pager_incidents
    rollback_trigger: "any gate fails for 1 hour OR pager fires"

  - name: general_availability
    percentage: 100%
    cleanup: "remove feature flag, delete old code path"