Experiment Priority System

Priority values determine the execution order of experiments in your backlog. Mastering the priority system ensures your most important tests run at the optimal time, maximizing the value of your testing program.

How Priority Works

Every experiment has a numeric priority value. When multiple experiments are ready to run, PressPlay selects the one with the highest priority number. It's that simple.

Priority Range

Priority values can be any positive integer. There's no fixed maximum, giving you flexibility to prioritize experiments as needed. In practice, most teams use a range of 1-100.

Example:

• Experiment A: Priority 75

• Experiment B: Priority 50

• Experiment C: Priority 85

Execution order: C (85) → A (75) → B (50)

Setting Priority

You assign priority when creating an experiment and can modify it anytime before the experiment starts running.

During Creation

1. Navigate to experiment creation

2. Configure experiment details (name, variants, settings)

3. Set the priority value in the priority field

4. Save the experiment

Modifying Priority

Change priority for experiments that haven't started yet:

1. Open the experiment details page

2. Click "Edit Experiment"

3. Update the priority value

4. Save changes

Important: You can only modify priority when the experiment is in READY, NOT_READY, or ERROR status. Once an experiment is IN_PROGRESS, STOPPING, or FINISHED, priority cannot be changed.

Strategic Priority Assignment

Business-Critical Tests

Assign the highest priorities (90-100) to experiments that:

• Address urgent performance issues

• Test time-sensitive opportunities (seasonal campaigns, feature launches)

• Block other planned experiments that depend on their results

• Have executive or stakeholder attention

High-Priority Optimization

Use 70-89 for experiments that:

• Target known pain points in your funnel

• Test hypotheses with strong supporting data

• Have significant potential impact on key metrics

• Fill gaps in your optimization roadmap

Standard Testing

Assign 50-69 to experiments that:

• Follow your regular testing cadence

• Test iterative improvements

• Explore new hypotheses with moderate confidence

• Maintain testing velocity

Low-Priority Exploration

Use 1-49 for experiments that:

• Test speculative ideas

• Explore long-term strategic questions

• Can wait for optimal timing

• Serve as learning exercises

Priority Best Practices

Use Spacing

Don't assign consecutive priority numbers. Instead, use increments of 5 or 10:

• Good: 90, 80, 70, 60

• Avoid: 90, 89, 88, 87

Spacing allows you to insert urgent experiments between existing ones without reorganizing your entire queue.

Document Priority Decisions

Add notes to your experiment explaining why it has a specific priority. This helps team members understand the rationale and make informed adjustments.

Regular Priority Reviews

Schedule weekly or bi-weekly reviews of your backlog priorities. Business priorities change, and your testing queue should reflect current needs.

Coordinate with Your Team

If multiple team members manage experiments, establish shared priority guidelines. Consistent prioritization prevents conflicts and maintains queue integrity.

Common Priority Patterns

Sequential Testing

When experiments build on each other's results:

• Foundation test: Priority 80

• Follow-up test A: Priority 79

• Follow-up test B: Priority 78

This ensures experiments run in the correct sequence even if creation happens out of order.

Parallel Hypothesis Testing

Testing multiple hypotheses for the same problem:

• Hypothesis A (strongest): Priority 75

• Hypothesis B (moderate): Priority 74

• Hypothesis C (exploratory): Priority 73

Results from higher-priority tests may inform whether lower-priority tests should still run.

Asset Type Organization

Group experiments by asset type using priority ranges:

• Icon experiments: 60-69

• Screenshot experiments: 70-79

• Description experiments: 50-59

This keeps similar experiments together in your queue for easier management.

Getting Maximum Priority

PressPlay provides an API endpoint to retrieve the current maximum priority value for your app. This helps you:

• Set a new experiment's priority higher than all existing experiments

• Understand the current priority range in use

• Automate priority assignment in custom workflows

Use this when you need to ensure an experiment runs immediately after the current test completes.

Priority and Experiment Status

Priority only matters for experiments in READY status. Experiments in other states are handled differently:

Handling Priority Conflicts

Tied Priorities

If multiple READY experiments have the same priority, the system uses creation timestamp as a tiebreaker. The older experiment runs first.

Priority Changes During Queue Processing

If you change priority while experiments are queued, the new priority takes effect immediately. The queue reorders automatically.

Advanced Priority Strategies

Dynamic Priority Adjustment

Adjust priorities based on incoming data:

• Promote experiments when supporting evidence strengthens

• Demote experiments when conditions change

• Reorder based on new business priorities

Priority Buffers

Reserve specific priority ranges for different scenarios:

• Emergency tests: 95-100

• Scheduled major tests: 80-90

• Standard queue: 40-75

• Backlog reserve: 1-35

Seasonal Priority Shifts

Adjust your priority strategy for high-traffic periods:

• Pre-season: Lower priorities for risky experiments

• Peak season: Higher priorities for proven optimizations

• Post-season: Higher priorities for exploratory testing

Next Steps

• Understanding the Backlog - How experiment queuing works in PressPlay

• Experiment Status Lifecycle - Status transitions and what you can edit at each stage

• Creating Your First Experiment - Step-by-step experiment setup