> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mavera.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Email Sequence A/B Testing

> Show top and worst-performing Close CRM cold emails to a Focus Group for qualitative A/B validation that explains the why behind engagement metrics

## Scenario

Your SDRs run email sequences in Close and track open/reply rates. You want to go beyond metrics: **take the top and worst-performing emails, show them to a Focus Group, and get qualitative reasoning** for why one works and the other doesn't. Synthetic A/B validation that explains the "why."

## Architecture

```mermaid theme={"dark"}
flowchart LR
    A[Close Email Activities] --> B["Top/worst performers"] --> C["POST /api/v1/focus-groups"] --> D["Persona-validated A/B insights"]
```

## Code

<CodeGroup>
  ```python Python theme={"dark"}
  import os, requests
  from requests.auth import HTTPBasicAuth
  from collections import defaultdict

  CLOSE_KEY = os.environ["CLOSE_API_KEY"]
  CLOSE_AUTH = HTTPBasicAuth(CLOSE_KEY, "")
  MAVERA_KEY = os.environ["MAVERA_API_KEY"]

  def close_get(path, params=None):
      r = requests.get(f"https://api.close.com/api/v1{path}", auth=CLOSE_AUTH, params=params or {})
      r.raise_for_status()
      return r.json()

  # 1. Pull sent email activities
  emails = [
      e for e in close_get("/activity/email/", {"_limit": 200}).get("data", [])
      if e.get("direction") == "outgoing" and e.get("subject")
  ]

  # 2. Deduplicate by subject, pick top and worst performers
  unique = list({e["subject"]: e for e in emails}.values())
  top_performers = unique[:3]
  worst_performers = unique[-3:]

  def fmt(e, label):
      return f"[{label}] Subject: {e['subject']}\nBody: {(e.get('body_text') or '')[:500]}"

  email_text = "\n\n---\n\n".join(
      [fmt(e, "TOP") for e in top_performers] + [fmt(e, "WORST") for e in worst_performers]
  )

  # 3. Run Focus Group
  fg_resp = requests.post(
      "https://app.mavera.io/api/v1/focus-groups",
      headers={"Authorization": f"Bearer {MAVERA_KEY}"},
      json={
          "title": "Close CRM Email A/B Validation",
          "personas": [
              {"name": "CTO at Series B Startup", "description": "Technical decision-maker, 50+ cold emails/day, values specificity."},
              {"name": "VP Sales at Mid-Market", "description": "Revenue-focused, skeptical of vendors, responds to ROI and proof."},
              {"name": "IC Developer", "description": "Bottom-up evaluator, hates marketing speak, wants docs and demos."},
          ],
          "questions": [
              f"Below are 6 cold emails — 3 TOP (high engagement) and 3 WORST (low engagement).\n\n{email_text}",
              "Which TOP email would you actually reply to? Why does it work?",
              "What makes the WORST emails easy to ignore or delete?",
              "If you could rewrite the worst email to be compelling, what would you change?",
              "What subject line patterns catch your attention vs. trigger spam instinct?",
          ],
      },
  )
  fg_resp.raise_for_status()
  fg = fg_resp.json()

  for resp in fg.get("responses", []):
      print(f"\n=== {resp['persona_name']} ===")
      for a in resp.get("answers", []):
          print(f"  Q: {a['question'][:80]}...\n  A: {a['answer'][:400]}\n")
  ```

  ```javascript JavaScript theme={"dark"}
  const CLOSE_KEY = process.env.CLOSE_API_KEY;
  const CLOSE_AUTH = "Basic " + Buffer.from(`${CLOSE_KEY}:`).toString("base64");
  const MAVERA_KEY = process.env.MAVERA_API_KEY;

  async function closeGet(path, params = {}) {
    const url = new URL(`https://api.close.com/api/v1${path}`);
    for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
    const res = await fetch(url, { headers: { Authorization: CLOSE_AUTH } });
    if (!res.ok) throw new Error(`Close ${res.status}: ${await res.text()}`);
    return res.json();
  }

  // 1. Pull sent emails
  const emails = ((await closeGet("/activity/email/", { _limit: 200 })).data || [])
    .filter((e) => e.direction === "outgoing" && e.subject);

  // 2. Deduplicate, pick top and worst
  const unique = Object.values(Object.fromEntries(emails.map((e) => [e.subject, e])));
  const top = unique.slice(0, 3);
  const worst = unique.slice(-3);

  const fmt = (e, label) => `[${label}] Subject: ${e.subject}\nBody: ${(e.body_text || "").slice(0, 500)}`;
  const emailText = [...top.map((e) => fmt(e, "TOP")), ...worst.map((e) => fmt(e, "WORST"))].join("\n\n---\n\n");

  // 3. Run Focus Group
  const fgRes = await fetch("https://app.mavera.io/api/v1/focus-groups", {
    method: "POST",
    headers: { Authorization: `Bearer ${MAVERA_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      title: "Close CRM Email A/B Validation",
      personas: [
        { name: "CTO at Series B Startup", description: "Technical decision-maker, values brevity." },
        { name: "VP Sales at Mid-Market", description: "Revenue-focused, responds to ROI and proof." },
        { name: "IC Developer", description: "Bottom-up evaluator, wants docs not marketing." },
      ],
      questions: [
        `Below are 6 cold emails — 3 TOP and 3 WORST.\n\n${emailText}`,
        "Which TOP email would you reply to? Why?",
        "What makes the WORST emails easy to ignore?",
        "How would you rewrite the worst to be compelling?",
        "What subject line patterns catch you vs. trigger spam instinct?",
      ],
    }),
  });
  const fg = await fgRes.json();
  for (const resp of fg.responses || []) {
    console.log(`\n=== ${resp.persona_name} ===`);
    for (const a of resp.answers || []) console.log(`  Q: ${a.question.slice(0, 80)}...\n  A: ${a.answer.slice(0, 400)}\n`);
  }
  ```
</CodeGroup>

## Example Output

```text theme={"dark"}
=== CTO at Series B Startup ===
  Q: Which TOP email would you reply to? Why?
  A: "Your API latency vs. Stripe's — 3 fixes." It's specific, implies
     homework on my stack, and promises actionable value. I'd open it
     because it signals technical depth, not generic outreach.

  Q: What makes the WORST emails easy to ignore?
  A: "Quick question" as a subject is an instant delete. Body starts
     with "I hope this finds you well" — signal this is a mass blast.
     No personalization, no value prop in sentence one.

=== IC Developer ===
  Q: How would you rewrite the worst to be compelling?
  A: Drop pleasantries. Lead with: "I noticed your team uses [tool] —
     here's a 2-line snippet that cuts deploy time by 40%." Link to
     docs, not a calendar invite.
```

## Error Handling

| Error                 | Cause                | Fix                                                     |
| --------------------- | -------------------- | ------------------------------------------------------- |
| `401 Unauthorized`    | Wrong auth method    | Use HTTP Basic — API key as username, empty password    |
| `429 Rate Limited`    | Exceeded 10-40 RPS   | Read `X-Rate-Limit-Reset` header; back off              |
| `body_text` is `null` | HTML-only emails     | Fall back to `body_html`, strip tags before Focus Group |
| No sequence data      | Emails sent manually | Group by subject similarity instead of `sequence_id`    |