[Q1-P0] Add integration tests against real production API

[karlwaldman]

Problem

We shipped SDK v1.4.1 with 100% timeout rate on historical queries. Unit tests passed because they used mocked responses.

Root Cause: No integration tests calling real API endpoints

Impact

• Severity: P0 - 100% failure rate for core feature
• Detection: Customer report (not CI/CD)
• Time to fix: 2 hours emergency fix
• Could have been: Hundreds of users affected

Solution

Add integration test suite that calls real production API:

# tests/integration/test_real_api_historical.py

@pytest.mark.integration
def test_1_year_query_against_production():
    """Test real 1-year historical query."""
    client = OilPriceAPI(api_key=os.getenv('TEST_API_KEY'))

    start_time = time.time()
    historical = client.historical.get(
        commodity='WTI_USD',
        start_date='2024-01-01',
        end_date='2024-12-31',
        interval='daily'
    )
    duration = time.time() - start_time

    # Would have caught the timeout bug
    assert duration < 120, f"Query took {duration}s, exceeds timeout"
    assert len(historical.data) > 300

Test Cases to Add

1. Endpoint existence tests

   • Verify all 4 historical endpoints exist (past_day, past_week, past_month, past_year)
   • Verify they return expected data format

2. Query completion tests

   • 1 day query completes successfully
   • 1 week query completes successfully
   • 1 month query completes successfully
   • 1 year query completes successfully

3. Timeout behavior tests

   • Queries complete within calculated timeout
   • TimeoutError raised if query exceeds timeout

4. Data validation tests

   • Correct number of records returned
   • Data format matches expectations
   • Pagination works correctly

Implementation

# New directory structure
tests/
  integration/
    __init__.py
    conftest.py          # Shared fixtures
    test_historical.py   # Historical endpoint tests
    test_prices.py       # Current price tests
    test_alerts.py       # Alert endpoint tests

CI Integration

# .github/workflows/test.yml
jobs:
  integration-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run integration tests
        env:
          TEST_API_KEY: ${{ secrets.TEST_API_KEY }}
        run: |
          pytest tests/integration/ --integration -v

Acceptance Criteria

• Integration test suite created in tests/integration/
• Tests cover all historical endpoints
• Tests validate timeout behavior
• Tests run in CI before every release
• Test API key configured in GitHub Secrets
• Tests pass consistently

Estimated Effort

Time: 2-3 hours
Complexity: Low (straightforward API calls)
Blocker: Need test API key with Production Boost access

Success Metrics

• Integration tests catch issues that unit tests miss
• Zero P0 bugs ship to production due to SDK issues
• Confidence in releases increases

Related Issues

• Historical timeout bug (idan@comity.ai)
• QA Assessment: sdks/python/QA_ASSESSMENT_HISTORICAL_TIMEOUT_ISSUE.md

Notes

This would have caught the historical timeout bug in CI before it reached customers. Must be completed before next SDK release.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

_{You can configure auto-planning by selecting labels in the issue_enrichment configuration.}

🔗 Similar Issues

Related Issues

• 🟡 P1: Optimize Historical Prices Query Performance - 60-90 Second Response Times oilpriceapi-node#8
• https://github.com/OilpriceAPI/oilpriceapi-api/issues/596
• https://github.com/OilpriceAPI/oilpriceapi-api/issues/608
• https://github.com/OilpriceAPI/oilpriceapi-api/issues/651

👤 Suggested Assignees

• karlwaldman

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!

[karlwaldman]

Closing as complete — the repo now has exactly this:

Integration tests against the real production API

• tests/integration/test_live_api.py, test_historical_endpoints.py (incl. the 1-year query that caused the v1.4.1 timeout bug), test_live_futures.py, test_live_subscriptions.py, test_demo_contract.py
• tests/contract/test_api_contract.py validates the response-schema assumptions

Run automatically in CI

• .github/workflows/live-tests.yml — every push/PR to main (2026-07-10 run: 9 live tests passed against production)
• .github/workflows/weekly-health.yml — Monday cron

Two weekly-health gaps found during verification (env-var mismatch skipping live futures/subscriptions tests; stale >300/>1000-point assertions vs the API's 100-point page cap) are fixed in PR #53.