diff --git a/AGENTS.md b/AGENTS.md index 45bd33c7..bf971121 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -95,6 +95,7 @@ longer exist — only some keep an alias. |---|---| | `wizard migrate statsig` | `migrate-statsig` | | `wizard migrate resend` | `migrate-email-resend` | +| `wizard migrate amazon-ses` | `migrate-email-amazon-ses` | **Commands vs. skills:** those audit subcommands **are** skills, promoted to commands via `cli: role: command`. A `role: skill` skill is reachable only via diff --git a/context/skills/migrate-email/config.yaml b/context/skills/migrate-email/config.yaml index 6f9a304f..22994778 100644 --- a/context/skills/migrate-email/config.yaml +++ b/context/skills/migrate-email/config.yaml @@ -16,3 +16,6 @@ variants: - id: resend display_name: Resend tags: [resend] + - id: amazon-ses + display_name: Amazon SES + tags: [amazon-ses] diff --git a/context/skills/migrate-email/references/amazon-ses/account.md b/context/skills/migrate-email/references/amazon-ses/account.md new file mode 100644 index 00000000..9c3b00ea --- /dev/null +++ b/context/skills/migrate-email/references/amazon-ses/account.md @@ -0,0 +1,172 @@ +# Amazon SES account + +What the migration needs from the person's Amazon SES account. The agent never changes the account: it reads what the person exports or allows. + +Amazon SES keeps identities, templates, and the suppression list per AWS Region. The client's Region setting names the Region a deployment sends from, or the environment it runs with, `AWS_REGION`, or `AWS_DEFAULT_REGION` for boto3; the question below asks for each deployment's. Each Region answers on its own host, `email..amazonaws.com`, so every command in this file sets `--region` to the Region it reads. `sdk-reference.md` names the setting each SDK picks the Region with. When the client also sets an endpoint, add `--endpoint-url` with that same URL to every command, so it reads the account the app sends with. + +The Regions belong to an AWS account, and each account keeps its own. Ask the person which deployments run the code, such as staging and production, which of them send into the PostHog project being migrated, and which AWS account and Regions each one's Amazon SES client sends with, by its account ID, per site when sites send with different credentials. Read each account only in the Regions its deployments send in. That is the account of the credentials the client is built with, or of the role it assumes, such as with `fromTemporaryCredentials` or `AssumeRole`, not the person's default CLI profile. The person reads it by running `aws sts get-caller-identity --query Account` where each deployment runs or with the role it assumes, or from that role's ARN. Check the IDs against the accounts the repository shows, such as the assumed role's ARN, a `TemplateArn`, or each deployment's account in Terraform or CDK config. Leave out an identity ARN for sending authorization, which names the identity's owner. When the IDs still differ after one more ask, or the person can't say which account a deployment uses, stop before reading and tell the person why. + +The moved code runs in every deployment, so a site counts once per deployment, with that deployment's account and Region, in every check `mapping.md` makes other than "Suppressions", such as its templates and configuration sets: it moves only when it would move in each. When the person can't give credentials to read a deployment's account, the sites it runs stay, the run goes on, and the reason is `its AWS account in not checked`. Under "Suppressions", each deployment that sends into the migrated project is its own moved site, and the import takes only their lists. The others go in the report, as "For the report" says. + +Read each account's Regions on their own. Before an account's reads, run `aws sts get-caller-identity --query Account`, which needs no permission, with the same credentials and without `--endpoint-url`, since it asks AWS STS, not Amazon SES, or ask the person to run it with theirs, and check that it prints that account's ID. When it prints another, stop and ask the person for that account's credentials: another account's list holds addresses Amazon SES never checked these sends against. + +## DNS records + +A domain verified at Amazon SES carries these records. Keep all of them until the cutover is done. The Amazon SES console shows the exact names under Identities, then the domain, for each Region. + +| Record | Name | What it does | +|---|---|---| +| `CNAME` (DKIM), three of them | `._domainkey`, each pointing to `.dkim..amazonses.com` or a similar name | Easy DKIM: signs Amazon SES's mail | +| `MX` | The custom MAIL FROM subdomain, such as `bounce` | Only when a custom MAIL FROM domain is set up: takes Amazon SES's bounces | +| `TXT` (SPF) | The custom MAIL FROM subdomain | Only when a custom MAIL FROM domain is set up: `v=spf1 include:amazonses.com ~all` | +| `TXT` (DMARC) | `_dmarc` | Optional, shared by every provider on the domain | + +- A domain set up with its own DKIM key (Bring Your Own DKIM) has one `TXT` record at `._domainkey` in place of the three CNAMEs. +- A domain verified in more than one Region has a set of DKIM records for each. +- PostHog's sender uses its own label for its MAIL FROM, `feedback` by default. Never give PostHog the subdomain Amazon SES uses as its custom MAIL FROM. +- PostHog's DKIM records have their own names, so they sit next to the Amazon SES `_domainkey` records. +- Keep the one `_dmarc` record. Never add a second one. +- Never add a second `v=spf1` record on one name. A name whose SPF record names `include:amazonses.com` keeps it: merge PostHog's include into that record. + +## Read-only key + +The app's AWS credentials, or the role it runs with, often reach other AWS services, and may not be allowed to read templates, configuration sets, or the suppression list. Ask the person to run the AWS CLI commands below with their own credentials and save the output, which needs no key from them. + +When the person chooses that you run them, ask them to create an IAM user whose policy allows only `ses:GetEmailTemplate`, `ses:GetTemplate`, `ses:GetEmailIdentity`, `ses:GetConfigurationSet`, `ses:GetConfigurationSetEventDestinations`, `ses:GetAccount`, `ses:GetIdentityNotificationAttributes`, and `ses:ListSuppressedDestinations`, and an access key for it. Ask them to store its key ID in `SES_READ_ACCESS_KEY_ID` and its secret in `SES_READ_SECRET_ACCESS_KEY` in the git-ignored env file. Run the commands with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` set from those two variables for that command only, `AWS_SESSION_TOKEN` and `AWS_SECURITY_TOKEN` unset for it, and `--region` set. The AWS CLI reads a session token from either of those next to the key pair, and AWS refuses a key sent with another key's token. The key reaches only the account it was made in: for each other account, the person runs the commands with their own credentials. Its last use is the suppression import in Step 5. Deleting it, when Step 5 asks, means deleting the access key and the IAM user in AWS and removing both variables from the env file. Never print the key, and never write it into another file. + +The steps call this key the full-access provider key. + +## Template content + +A site whose template moves needs the template's content: Step 3 checks it, and Step 7 brings it into the code, or the person makes a template in PostHog from it when they ask for one. The API returns the content: the subject, the HTML, and the text, each as a string. A site that sends an inline template has its content in the code: read it there. + +A template belongs to one account and Region. Read each one with its site's account's credentials, in the Region its site sends in, or, for a site that names a `TemplateArn`, in the Region the ARN names. + +**Read with the AWS CLI (preferred).** Ask the person to run, for a v2 site: + +```bash +aws sesv2 get-email-template --template-name --region > +``` + +The answer holds `TemplateName` and `TemplateContent`, which is `{ Subject, Html, Text }`. A site that sends with the v1 API (`SendTemplatedEmail`) uses: + +```bash +aws ses get-template --template-name --region > +``` + +The answer holds `Template`, which is `{ TemplateName, SubjectPart, HtmlPart, TextPart }`. + +Each part is a JSON string: read it decoded, so `\n` is a line break and `\"` a quote. A part the template doesn't have is missing from the answer. + +Ask the person where they saved each file. + +**Read from the API.** With the key above, run the same command yourself and keep the answer only in the temporary file Step 3 writes, which Step 7 deletes once it has brought the email into the code. The key is deleted at the end of Step 5, so when Step 7 finds that file gone, ask the person to run the AWS CLI command above for that template and tell you where they saved it. + +Read the person's copies where they saved them. Never write the content into the plan or the report. + +## Configuration sets + +A configuration set decides, with the account's VDM settings, whether Amazon SES tracks opens and clicks, and it decides where a send's events go and which suppression reasons apply. The reads below are how to tell whether a site's emails had an open pixel, for Step 3's "Opens the provider didn't track", as `mapping.md` says under "Open and click tracking". `mapping.md` needs the sets the sites use: each set a site names in `ConfigurationSetName`, and the default set of each identity the sites send from. + +**Check the CLI first.** An AWS CLI older than a setting can print `null` for it. In the same request as the set and account reads below, those under "Suppression list" included, have the CLI that runs them, the person's when they run them, print the input shape of the calls that set those fields, which sends no request: + +```bash +aws sesv2 create-configuration-set --region --generate-cli-skeleton input +aws sesv2 put-account-suppression-attributes --region --generate-cli-skeleton input +aws sesv2 put-account-vdm-attributes --region --generate-cli-skeleton input +``` + +The first must name `VdmOptions`, `MessageSecurityOptions`, `ArchivingOptions`, `ValidationOptions`, and `SuppressionScope`, the second `ValidationAttributes`, and the third `VdmAttributes`. An error that isn't about an unknown option or command is not a missing field: fix what it names, such as a Region, and run it again. When one is missing, ask the person to update the AWS CLI, and use none of the reads until it names them all. When the person can't update it, every site stays, and the reason is `its Amazon SES settings not checked`. + +**Read with the AWS CLI (preferred).** Ask the person to run, in each Region, for each domain or address the sites send from: + +```bash +aws sesv2 get-email-identity --email-identity --region --query '{Set: ConfigurationSetName, Policies: Policies}' +``` + +`Set` is the identity's default configuration set, or `null` when it has none. `Policies` maps each of the identity's sending authorization policies to its JSON, which `mapping.md` reads under "Sending turned off". Then, for each set the sites name and each default set: + +```bash +aws sesv2 get-configuration-set-event-destinations --configuration-set-name --region +aws sesv2 get-configuration-set --configuration-set-name --region --query '{Sending: SendingOptions, Suppression: SuppressionOptions, Vdm: VdmOptions, Delivery: DeliveryOptions, Security: MessageSecurityOptions, Archiving: ArchivingOptions}' +``` + +The first answer holds `EventDestinations`, each with `Enabled` and `MatchingEventTypes`: a set tracks opens when an enabled destination lists `OPEN`, and clicks when one lists `CLICK`. In the second, `Sending.SendingEnabled` is `false` when sending is turned off for the set: Amazon SES refused its sends, and its sites stay unless the person agrees to deliver them through PostHog, as `mapping.md` says under "Sending turned off". `Suppression` is the set's own suppression settings, or `null` when it uses the account's, and `{}` or an empty `SuppressedReasons` when the set turns suppression off. A `SuppressionScope` or `ValidationOptions` without `SuppressedReasons` overrides only where the addresses are kept or Auto Validation, not the reasons. `Suppression.ValidationOptions`, when present, overrides the account's Auto Validation for the set's sends, on when its `ConditionThreshold.ConditionThresholdEnabled` is `ENABLED` and off when it is `DISABLED`. `Delivery.TlsPolicy` is `REQUIRE` when the set delivers only over TLS, and `Delivery.MaxDeliverySeconds`, when below 50400, is how soon it gives up delivery. `Security.SigningScheme.SmimeScheme`, when set, has Amazon SES sign the set's sends with S/MIME; `DefaultScheme` adds no signature. `Archiving.ArchiveArn`, when set, archives them in a Mail Manager archive. Each of these keeps its sites, as `mapping.md` says. `Vdm` is the set's own VDM settings, or `null` when it uses the account's. Its `DashboardOptions.EngagementMetrics`, when set, turns VDM engagement tracking on or off for the set's sends. + +Then read the account's VDM settings, in each Region: + +```bash +aws sesv2 get-account --region --query VdmAttributes +``` + +The answer holds `VdmEnabled` and `DashboardAttributes.EngagementMetrics`, or is `null` when the account never turned VDM on. VDM engagement tracking tracks the opens and clicks of every send whose set doesn't override it, and of every send with no set. + +Then read whether the account sends at all, in each Region: + +```bash +aws sesv2 get-account --region --query '{Sending: SendingEnabled, Enforcement: EnforcementStatus, Production: ProductionAccessEnabled}' +``` + +`Sending` is `false` when sending is off for the account, turned off by the person or paused by AWS. `Enforcement` is `SHUTDOWN` when AWS stopped the account's sending. `Production` is `false` when the account is in the sandbox, where Amazon SES sends only to verified addresses. Each of these keeps the sites that send in that Region unless the person agrees to deliver them through PostHog, as `mapping.md` says under "Sending turned off". Ask the person to paste the answers, or where they saved them. + +Then read where each identity sends its SNS notifications, in each Region, since `get-email-identity` doesn't return them: + +```bash +aws ses get-identity-notification-attributes --identities --region +``` + +The answer's `BounceTopic`, `ComplaintTopic`, and `DeliveryTopic` name the SNS topics that get the identity's bounces, complaints, and deliveries, and a topic left out sends none. A handler subscribed to one of them gets the events of every send from that identity, as `mapping.md` says under "Webhooks". When this can't be read, a handler of SNS notifications that matches by message id or by a tag counts as fed by every identity the sites send from. + +**Read from the API.** With the key above, run the same commands yourself. + +## Suppression list + +Amazon SES keeps an account-level suppression list in each Region, with a reason of `BOUNCE` or `COMPLAINT` for each address. The account's settings name the reasons it suppresses. A configuration set can override them, for the sends that use it, with its own reasons or with none. Amazon SES accepts a send to an address on the list and doesn't send it only when the entry's reason applies to the send: its set's reasons when the set overrides the account's, and the account's otherwise. An entry with another reason is still sent to. An account that never turned the list on has an empty one. Amazon SES adds an address that hard-bounced or complained, and people add addresses by hand too: in the Amazon SES console (Configuration, then Suppression list, Add email address or Bulk actions), with `aws sesv2 put-suppressed-destination`, or with an import job, and the app with the calls `mapping.md` names under "Calls that change the suppression list". An address added by hand also carries `BOUNCE` or `COMPLAINT`, so the list doesn't tell it apart. Step 3's question about the sites that rely on the suppression list leaves out each site whose sends Amazon SES checked against no reason, its set's or the account's, as "Map each entry" works them out: Amazon SES sent to every address on the list for it. + +Read the account's suppression settings first, in each Region: + +```bash +aws sesv2 get-account --region --query SuppressionAttributes +``` + +The answer's `SuppressedReasons` are the account's reasons. Its `ValidationAttributes.ConditionThreshold.ConditionThresholdEnabled` is `ENABLED` when Auto Validation is on for the account, which `mapping.md` covers under "Auto Validation". + +**Read with the AWS CLI (preferred).** Ask the person to run, for each Region: + +```bash +aws sesv2 list-suppressed-destinations --region --output json > +``` + +The answer holds `SuppressedDestinationSummaries`, a list of `{ EmailAddress, Reason, LastUpdateTime }`, and `NextToken` when more entries follow. The CLI doesn't fetch the next page itself: while the answer has a `NextToken`, ask them to run the command again with `--next-token ` into a new file. Wait and retry on `TooManyRequestsException`. Ask the person where they saved each file. + +**Read from the API.** With the key above, run the same commands yourself, page by page. + +**Map each entry.** Work out the reasons that apply to each moved site: the `SuppressedReasons` of its configuration set, the one it names or its identity's default one, when the set's `SuppressionOptions` holds that list, with no reason when the list is empty or `SuppressionOptions` is `{}`. A set that overrides only `SuppressionScope` or `ValidationOptions`, or both, keeps the account's reasons, and so does a set whose `SuppressionOptions` is `null`. Import the entries whose reason applies to at least one moved site that sends with that list's account in its Region, and skip the rest. Add the lists and reasons an earlier import took, as "An earlier import" says below. An address can sit on more than one of these lists with a different reason on each. Before the first batch, merge the entries of every list the import takes into one per address, compared without case, and give it `bounce` when any of them is `BOUNCE`: PostHog keeps the first entry it gets for an address, in a batch or across batches, and skips the rest, so a `complaint` sent first would let moved sends reach an address Amazon SES suppressed for a bounce. PostHog keeps one suppression list per project and checks it on every moved send. A moved site whose own reasons leave out `BOUNCE` while another moved site imports `BOUNCE` entries, or that sends through another account or Region than the list a `BOUNCE` entry came from, has its sends to those addresses refused where Amazon SES sent them: it moved only with the person's yes, as `mapping.md` says under "Suppressions". + +| Amazon SES reason | PostHog `reason` | +|---|---| +| `BOUNCE` | `bounce` | +| `COMPLAINT` | `complaint` | + +PostHog refuses a moved send only to an address it holds as a `bounce`, as Step 5 says: a `complaint` entry blocks only PostHog's marketing email. So wherever this file and `mapping.md` say that PostHog refuses a site's sends for a list or a reason it imports, only `BOUNCE` entries count, and a list or reason that adds only `COMPLAINT` entries widens nothing. The import still takes the `COMPLAINT` entries whose reason applies, and the report and a later run record them like the others. + +**An earlier import.** A moved send no longer names its Region or configuration set, so once a run has moved sites, what its import took can't be worked out from the code again. An import takes from one list per AWS account and Region: two accounts' lists in one Region are two lists. Name each by its account ID, read as the top of this file says, and its Region. An earlier run of this app shows as its report, `posthog-migrate-email-amazon-ses-report.md` at the project root, or as a PostHog client with the provider name set to `amazon-ses`, under the name the rewrite file gives, such as `migratedFrom: 'amazon-ses'` in Node.js or `migrated_from="amazon-ses"` in Python. Outside a re-import, also ask the person once whether a migration from Amazon SES, from this app or another, imported into the PostHog project the app sends from beyond what this repository's report records. Take the lists and reasons each earlier import took from its report, the ones it records under "Suppression list" as taken by the import, never a deployment's it records as not imported, and ask the person for what no report holds. Carry the deployments it records as not imported into this run's report, as "For the report" says, with the account ID, Regions and reasons it records: they belong to the sites the earlier run moved, which no longer send with Amazon SES, so a change to the deployment's AWS account doesn't change them. Take only the project from this run's answers about the deployments, and drop a deployment that no longer runs the code. One that now sends into the migrated project stays under "Not imported" too, its record unchanged. Next to the record, each run works out two sets again, once `mapping.md` settles which sites move, by comparing the record both ways with what this run's import and the earlier ones take: the recorded lists and reasons that neither takes, which the person imports with the re-import "For the report" gives under "Not imported", and the ones they take beyond the recorded ones, which PostHog now refuses its sites' sends for. It drops out when both are empty. + +- A re-import, as Step 5 says under "Re-import only", always follows a run, and takes exactly the lists and reasons its report records under "Suppression list" as taken by the import, never a deployment's it records as not imported, reading each with credentials whose account ID matches it, as the top of this file says. When the report is gone or has no "Suppression list" section, ask the person; when they can't say, stop before importing and tell the person why. +- A re-import of a deployment the report records under "Not imported", which the person asks for by that deployment and its AWS account ID, as "For the report" gives it, takes the lists and reasons the report records for that deployment, all of them every time, since each list keeps growing. It imports into the project the report records for that deployment: in Step 5's project check under "Re-import only", ask whether the MCP's active project is that one, naming it, instead of whether production sends from it, switch to it as that check does when it isn't, and stop before importing when it isn't listed. Before importing, ask the person once whether another app's Amazon SES migration imported into that project. Then gather the sites whose sends the import can refuse where Amazon SES delivered them: this deployment's sites whose sends its own entries refuse, worked out over every site its record lists as `mapping.md` says under "Suppressions", which happens when they send through more than one list or not all of them suppress `BOUNCE`; the sites of every other entry the report records under "Not imported" as sending into that project, the same deployment's entry for another AWS account included; when it is the migrated project, this app's moved sites, named as `mapping.md` says under "Suppressions", by the code whose PostHog client carries the provider name `amazon-ses` and the report's "Send call sites" rows whose status starts with "Moved"; and the other app's, when the person named one. Keep only the sites that this record's `BOUNCE` entries reach beyond what they sent through: a list other than their own, by their recorded list or the lists the import took for them, or `BOUNCE` where they don't suppress it, as `mapping.md` says under "Suppressions". A site whose lists or reasons neither the report nor the person can give counts as reached, as an unread set does there. When there are none, import. Otherwise ask once whether PostHog may refuse those sends, naming the sites per deployment, and saying that a no leaves this deployment's moved sends reaching the addresses its Amazon SES suppressed for a bounce. Without a yes, stop before importing and tell the person why. Ask again on every re-import that finds such sites: the answer is not recorded, and what the project holds changes. +- A later run imports the earlier lists and reasons too, alongside its own, and checks the widening both ways, as `mapping.md` says under "Suppressions": PostHog's list already holds the earlier entries, and the sites the earlier runs moved get every entry this run adds. When neither a report nor the person can say what an earlier import took, every site this run moves counts as one whose reasons differ, and everything it imports counts as widening the list for the earlier sites. + +**Leave out the other lists.** + +- **The global suppression list** is kept by Amazon SES itself, not by the account, so there is nothing of the person's to import. +- **Contact list unsubscribes** apply only to sends with `ListManagementOptions`, and those sites stay on Amazon SES. + +## For the report + +Under "Taken by the import": each list the import took, this run's and every earlier one's, by AWS account ID and Region, with the reasons that applied to a moved site sending there, whether or not the list held entries for them, so a later run or re-import takes the same. + +When the person agreed that this run's import adds a list or a reason for the sites earlier runs moved: those sites, by file and line, and the lists and reasons added. PostHog now refuses their sends to those addresses where Amazon SES sent them. + +Under "Not imported": each deployment that runs the moved code but sends into another PostHog project, and each one an earlier report records there, carried as "An earlier import" says, with that project, its account ID, and the Regions and reasons that applied to its sites, worked out as "Map each entry" does for this run's and merged with the earlier report's for the same deployment, project, and account ID, as "An earlier import" carries it. With each, list every one of its sites, by file and line, with the list, by account ID and Region, and the reasons that applied to it. A carried entry keeps its earlier sites, as recorded, and adds this run's. None of these entries was imported into that project, so that deployment's moved sends reach the addresses Amazon SES suppressed for a bounce until the import tool adds them there as `bounce` entries. An address the person adds by hand in PostHog (Workflows, then Messaging, then Suppression list) is a manual entry, which blocks only PostHog's marketing email, so it doesn't stop them. For a carried one in the migrated project, that holds only for the first of its two sets; the import below takes its whole record, which is safe to repeat. Give the prompt that imports them, `Re-import the Amazon SES suppression list into PostHog with the migrate-email-amazon-ses skill, for the deployment with AWS account , which sends into `, which reads them with that account's credentials, asks before it widens that project's list, and imports into that project, as "An earlier import" says. Ask for it right before the code deploys there, or now for a carried deployment, whose code already sends there, then weekly while any site stays on Amazon SES. When two such deployments send into one project with different lists, or a carried one now sends into the migrated project, adding its entries refuses mail Amazon SES sent for the others there: say which. For a carried one in the migrated project, also name the lists and reasons the import took beyond its recorded ones: PostHog refuses its sites' sends to those addresses where Amazon SES delivered them, and nothing in the app shows it. + +Each site whose note says `suppressions widened: the person agreed`: PostHog now refuses its sends to the addresses the import took from another account's or Region's list, or for a reason its configuration set doesn't suppress, where Amazon SES sent them. diff --git a/context/skills/migrate-email/references/amazon-ses/mapping.md b/context/skills/migrate-email/references/amazon-ses/mapping.md new file mode 100644 index 00000000..0408fe87 --- /dev/null +++ b/context/skills/migrate-email/references/amazon-ses/mapping.md @@ -0,0 +1,244 @@ +# Amazon SES to PostHog mapping + +This file decides, field by field, whether PostHog can send an Amazon SES call site. A site moves only when every field it sets moves. Otherwise it stays on Amazon SES, and the reason names the first field or feature that keeps it there. `api.md` has the PostHog side in full, and the rewrite file for the site's language, which Step 7 names, has PostHog's names in that language. + +PostHog fields are written in their Node.js names here, such as `replyTo`. Use the name the rewrite file gives for the site's language, such as `reply_to` in Python. Amazon SES fields are written in their v2 names, with the v1 name after a slash where it differs. `sdk-reference.md` has both. + +## Fields + +| Amazon SES | PostHog | A site stays on Amazon SES when | +|---|---|---| +| `FromEmailAddress` / `Source` | `from` | Never. `Name
` stays as it is. When the code builds it from a name and an address, put the name in double quotes when it holds a comma, a quote, or another special character. It needs a verified sender (Step 4) | +| `Destination.ToAddresses` | `to` | It holds more than one address, or none. A list with one entry moves as that entry. `Name
` moves as the bare address: note in the plan row that the To line loses the name | +| `Destination.CcAddresses`, `BccAddresses` | none | Always. PostHog's send doesn't take it yet | +| `ReplyToAddresses` | `replyTo`, one address or a list | An entry carries a name, there are more than 10, or the code builds the value at run time and you cannot tell that it holds bare addresses only. A value read from an environment variable or a config file counts as read when `.env.example`, the config file, or the person shows a bare address for it | +| `Subject.Data` | `subject` | Never | +| `Body.Html.Data`, `Body.Text.Data` | `html`, `text` | Never | +| `Charset` on the subject or a body part | none | It is set to anything other than UTF-8. Ignore case, `-`, and `_` when you compare: `utf-8`, `utf8`, and `UTF_8` are UTF-8 and move. Any other charset, such as `ISO-8859-1`, keeps the site | +| `Content.Template` with `TemplateName` or `TemplateArn` and `TemplateData` / `SendTemplatedEmail` with `Template` or `TemplateArn` and `TemplateData` | the template's email, brought into the code and rendered with `TemplateData`, sent as `subject`, `html`, and `text`. A template made in PostHog, by its `templateId` with `templateData`, only when the person asks for one | See "Templates" | +| `Content.Template.TemplateContent` with `TemplateData` | none: the code renders `TemplateData` into its `Subject`, `Html`, and `Text` before the send | It uses a tag other than `{{name}}` or a path such as `{{contact.firstName}}`, as "Templates" says | +| `Content.Raw` / `SendRawEmail` | none | Always: the code builds the MIME message itself | +| `Attachments` | none | Always. PostHog's send doesn't take it yet | +| `Headers` | none | Always: no PostHog SDK takes headers yet | +| `EmailTags` / `Tags` | none | Always: no PostHog SDK takes tags yet | +| `ConfigurationSetName`, or the sending identity's default configuration set when the send names none | none | A webhook handler matches the set's events by message id or by a tag (see "Webhooks"), or the set's `DeliveryOptions.TlsPolicy` is `REQUIRE`, or its `DeliveryOptions.MaxDeliverySeconds` is below 50400, the most Amazon SES allows, or it has a `MessageSecurityOptions.SigningScheme.SmimeScheme` or an `ArchivingOptions.ArchiveArn`: Amazon SES delivered it only over TLS, gave up after that time, signed it with S/MIME, or archived it in Mail Manager, and PostHog's send has none of these. A set whose sending is off keeps the site only until the person agrees to deliver through PostHog, as "Sending turned off" says. Otherwise it moves: note the set's name in the plan row, because its event destinations stop getting events for this site. `account.md` says how to read an identity's default set | +| Open and click tracking: `ConfigurationOverrides.Tracking`, the configuration set's event destinations and VDM options, or the account's VDM settings | none: PostHog adds its open pixel, and counts no clicks for a sensitive send: the code form, or a template made in PostHog that is marked sensitive | Its emails had no open pixel and the person didn't agree to add one, or nobody can tell whether they had one, as Step 3 says under "Opens the provider didn't track". See "Open and click tracking" below. `OpenTrackingEnabled` or `ClickTrackingEnabled` set to `DISABLED` counts as set: it turns that tracking off for the send, whatever the set or the account does | +| `ListManagementOptions` | none | Always: no PostHog SDK takes it | +| `FeedbackForwardingEmailAddress` / `ReturnPath` | none | Always: no PostHog SDK takes it | +| `FromEmailAddressIdentityArn`, `FeedbackForwardingEmailAddressIdentityArn` / `SourceArn`, `ReturnPathArn`, `FromArn` | none | Always: the site sends for an identity another AWS account owns | +| `EndpointId` | none | Always. PostHog's send doesn't take it yet | +| `TenantName` | none | Always. PostHog's send doesn't take it yet | + +**Open and click tracking.** PostHog adds its open pixel to every email it sends, to count opens, and counts no clicks for a send in the code form, as Step 7 says. No field changes this: `captureEngagement` decides only whether the email's events land on the recipient's person, so tracking never maps to it. Amazon SES decides opens and clicks one at a time. When the send's `ConfigurationOverrides.Tracking` sets `OpenTrackingEnabled` or `ClickTrackingEnabled`, that value decides, with one limit: `DISABLED` doesn't track it, and `ENABLED` tracks it when the send's configuration set has an event destination that publishes any event type, VDM engagement metrics are on for the send, or the other one of opens and clicks is tracked for the send. Without any of these, AWS ignores `ENABLED` and the message is sent untracked. Otherwise Amazon SES tracks it when either of these is on for the send: + +- VDM engagement tracking, which tracks both opens and clicks. The send's configuration set decides it when the set's `VdmOptions.DashboardOptions.EngagementMetrics` is set. Otherwise the account's `VdmAttributes.DashboardAttributes.EngagementMetrics` decides, for a send with no set too. Either is on only when it is `ENABLED` and the account's `VdmAttributes.VdmEnabled` is `ENABLED`. +- An enabled event destination of the send's configuration set, the one it names or its identity's default one, whose `MatchingEventTypes` lists `OPEN` for opens or `CLICK` for clicks. + +A send with neither gets no tracking pixel, and its links stay as written. Read the account's VDM settings and each set the sites use, as `account.md` says under "Configuration sets", and work out, for opens and for clicks, whether Amazon SES tracked them: + +- Opens tracked: the site moves. +- Opens untracked: the site's emails had no open pixel. It moves only when the person agrees that PostHog may add one, as Step 3 says under "Opens the provider didn't track", and otherwise stays, and the reason is `open tracking off`. +- Clicks, for every site that moves, whatever its opens: when Amazon SES tracked its clicks, or its click tracking isn't checked, and the site moves in the code form, or to a template made in PostHog that is marked sensitive, note in the plan row that PostHog counts no clicks for it. When Amazon SES didn't track its clicks, or its click tracking isn't checked, and it moves to a template made in PostHog, ask the person to mark that template sensitive, as Step 3 says under "A template in PostHog". + +When a site's open tracking depends on a setting the person can't give and the read-only key can't read, such as the account's VDM settings or a set's event destinations, nobody can tell whether its emails had an open pixel: the site stays, and the reason is `open tracking not checked`, as Step 3 says under "Opens the provider didn't track". + +An Amazon SES send has no idempotency key, so the moved site sends none either. PostHog's SDK makes one for each call and reuses it on its own retries. + +## Calls + +| Amazon SES call | A site stays on Amazon SES when | +|---|---| +| `SendEmail` in any SDK: `SendEmailCommand`, `AWS.SES().sendEmail`, `AWS.SESV2().sendEmail`, `send_email`, `sendEmail`, `SendEmail`, `SendEmailAsync`, `send_email()`, `ExAws.SES.send_email`, `ExAws.SES.send_email_v2`, `POST /v2/email/outbound-emails`, `Action=SendEmail` | A field above keeps it | +| `SendTemplatedEmail` / `send_templated_email` | A field above keeps it, or "Templates" does | +| `SendRawEmail` / `send_raw_email`, and `SendEmail` with `Content.Raw` | Always: the code builds the MIME message itself | +| `SendBulkEmail`, `SendBulkTemplatedEmail`, and their names in each SDK | Always: each sends several emails in one call | +| Nodemailer's `SES` transport | Always: PostHog has no Nodemailer transport | +| SMTP through `email-smtp..amazonaws.com` | Always here | +| A framework mailer (Django, Rails, Laravel, Symfony, Swoosh, Bamboo) | Always: PostHog has no mailer driver | +| A send in a language without a PostHog SDK that sends email, such as Java or Kotlin | Always. Step 7 lists the languages PostHog's SDKs cover | + +## Code around the send + +Most AWS SDKs throw on an error: Node.js, Python, Ruby, PHP, .NET, and Java. The Go, Rust, and Elixir SDKs return an error value. The rewrite file says under "The answer" how PostHog's SDK reports an error in its language. Keep the site failing the same way it did, as the Rewrite section says. + +| Code | A site stays on Amazon SES when | +|---|---| +| A `catch`, `except`, `rescue`, or error check that only logs the error, returns, or throws it again | Never. Log PostHog's error instead | +| A branch on an Amazon SES error name, such as `err.name === 'MessageRejected'`, `e.response['Error']['Code']`, `e.code`, `getAwsErrorCode()`, `ErrorCode`, `errors.As(err, &mr)` with `*types.MessageRejected`, or a `SendEmailError` variant | The name is anything but `TooManyRequestsException`, the v1 API's `Throttling`, or `BadRequestException`. A branch on `TooManyRequestsException` or `Throttling` moves as a check of PostHog's status code for `429`, and a branch on `BadRequestException` moves as a check of PostHog's error name `validation_error` | +| A branch on the HTTP status, such as `err.$metadata.httpStatusCode`, `e.response['ResponseMetadata']['HTTPStatusCode']`, `getStatusCode()`, `StatusCode`, `HTTPStatusCode()`, or `raw_response()` in Rust | Never. A branch on `400` reads PostHog's error name `validation_error`, and also its status code for `429` when the site sends with the v1 API (see "Errors"). A branch on another status, such as `429`, reads PostHog's status code | +| The `MessageId` stored for a webhook handler or a `GetMessageInsights` read | Always, when a handler or the read uses it | +| The `MessageId` logged | Never. Log PostHog's message id | +| The `MessageId` returned to the site's callers or stored for the app's own use | When the site throws, retries, or reports a failure on any error, unless its callers already handle a send without an id: a send PostHog refuses as suppressed has none, as "Rewrite" says. Otherwise never: use PostHog's message id | +| A deadline, timeout, or retry setting on the request or its client | As "HTTP" below says | + +## HTTP + +A site whose Amazon SES request had its own deadline keeps it, as `7-rewrite.md` says under "Set up the client". Only a raw HTTP request that never retried can have one that PostHog's SDK keeps: a timeout the code set on its HTTP client, such as `AbortSignal.timeout(5000)` on `fetch`, `got`'s `timeout.request`, a `timeout` of one number in `requests` or `httpx`, `'timeout'` in Guzzle, a `Net::HTTP` or Faraday timeout that `rewrite-ruby.md` says under "A deadline" is the SDK's kind of limit, Go's `http.Client` `Timeout` or a context with a deadline, and .NET's `HttpClient.Timeout` or a cancellation token with a timeout. Step 7 and the rewrite file's "A deadline" say which limits count and which the SDK can't keep, such as `axios`'s or `ky`'s `timeout`, or one on connecting alone, and those keep the site. + +The AWS SDKs and `aws4fetch`'s `AwsClient` retried inside their deadlines and timeouts, each in its own way, and a raw request can retry through its HTTP client. The moved send would retry differently, so each of these keeps the site: `moves: no — the PostHog SDK can't keep the request's deadline`, or `moves: no — the PostHog SDK can't keep the request's retries` for a retry setting without a timeout. + +- A deadline on an AWS SDK or `AwsClient` call, such as `{ abortSignal: AbortSignal.timeout(5000) }`, a Go context with a deadline, or a .NET cancellation token with a timeout. +- A timeout or retry setting in the AWS client's config, such as `requestTimeout`, `maxAttempts`, `read_timeout`, `connect_timeout`, `retries`, `http_read_timeout`, `retry_limit`, PHP's `'http' => ['timeout' => ...]`, Rust's `timeout_config` or `retry_config`, `ex_aws`'s `recv_timeout`, or `AwsClient`'s `retries`. +- A raw request's HTTP client that retries the POST, such as `got` or `ky` with `POST` in `retry.methods`, `axios-retry`, Faraday's `retry` middleware with `:post` in `methods` or a `retry_if`, a Guzzle retry middleware, or a `requests` adapter with a `Retry`. + +Follow a context, token, or abort signal back through the callers, as for the distinct ID. One that any caller builds from a timeout, such as `AbortSignal.timeout` or an `AbortSignal.any` that holds one, is a deadline. One that only cancels, such as a request's, is no deadline, and neither is a deadline the runtime or framework puts on the request, such as Lambda's or an incoming gRPC call's. When the callers pass different deadlines, or some pass none, and the send can't take the signal, context, or token, the site stays with the deadline reason. Pass it where the send takes one, as Go's context and .NET's token do. Where the send takes none, as in Node.js, the send goes on after a cancel: note it in the plan row. + +## Templates + +A site that sends a stored template moves by bringing the template into the code, as `template-to-code.md` says. The check needs the template's content, because the code has only its name or ARN. `account.md` says how to get it, from the account and Region the site sends with. The code renders the template's subject, HTML, and text with the values of the site's `TemplateData`, and the send sets the rendered `subject`, `html`, and `text`. + +A site that sends an inline template, `Content.Template.TemplateContent`, already has its content in the code: Amazon SES only filled in the values. It moves the same way, from the content the code passed. Literal strings become the email, as `template-to-code.md` says under "Where the email goes". Content the code reads at run time, such as from a file or a database, stays where it is, and a function next to the send replaces each marker in it. Amazon SES takes only simple substitutions in an inline template, so one that uses anything other than `{{name}}` or a path such as `{{contact.firstName}}` stays: the docs don't say what Amazon SES did with the rest. + +`TemplateData` is a JSON string. The code renders from the object it was built from, before `JSON.stringify`, `json.dumps`, `Jason.encode!`, or the language's own encoder. When the code only has the string, such as one read from a queue, parse it first. + +Amazon SES templates are Handlebars. In the code: + +| Amazon SES | In the code | +|---|---| +| `{{name}}`, `{{ name }}` | The value. Amazon SES inserted it as it is, unescaped, in every part. In the HTML, escape it anyway, except a value the code builds as HTML itself, under the rule of `template-to-code.md` | +| `{{contact.firstName}}` | The nested value, the same way | +| `{{{name}}}` | The same as `{{name}}`: Amazon SES escapes neither | +| A value missing from `TemplateData`, or a missing step of a path | Amazon SES failed the render: it accepted the send, answered a `MessageId`, and never sent it, with a `Rendering Failure` event that says `Attribute '' is not present in the rendering data.` The render in the code raises before the send, so the email still doesn't go out, and the site's error handling now sees it. Note it in the plan row | +| A `null` value, or `""` | The docs name only a missing value. Treat `null` as missing, and render `""` as nothing. Note in the plan row each value the code can pass as `null` or `""` | +| A number or a boolean | As `TemplateData` writes it, such as `10.0`, `3`, or `true`: Python's `True` renders as `true`. The docs don't say how Amazon SES writes a number, so render the text of the JSON | +| `{{#if a}}…{{else if b}}…{{else}}…{{/if}}` | A branch that tests each value for Handlebars truth, in order. With `includeZero=true`, `0` is true | +| `{{#unless a}}…{{else}}…{{/unless}}` | The same branch, with the test turned around | +| `{{#each items}}…{{else}}…{{/each}}` | A loop over the list, or over the values of an object in the order its keys stand in `TemplateData`. The `{{else}}` part shows when there is nothing to walk | +| Inside `{{#each}}`: `{{this}}`, `{{this.name}}`, or a bare `{{name}}` | The current item, or its field. A bare name reads only the item. When an item can lack that field, the template stays: the docs don't say whether Amazon SES failed the render or read the value outside the loop | +| Inside `{{#each}}`: `{{@index}}`, `{{@key}}`, `{{@first}}`, `{{@last}}`, `{{../name}}`, `{{@root.name}}` | The item's position from 0, its key in an object, whether it is the first or the last, and the value one level out or at the top | +| `{{#with contact}}…{{else}}…{{/with}}` | The block, where bare names read the fields of `contact`, or the `{{else}}` part when `contact` is empty: `false`, missing, `null`, `""`, or an empty list. This is not the truth test of `if`: the Handlebars docs list `0` as false for `if` but say only "empty" for `with`, so they don't settle `0`. When the value can be `0`, the template stays, and the reason is `its {{#with}} can take 0` | +| `{{lookup obj key}}`, `(lookup items 0)` | The entry of `obj` under the value of `key`, or the list's entry at that position. The docs don't say what a missing entry renders: treat it as a missing value | +| `{{#* inline "name"}}…{{/inline}}` with `{{> name}}` | A small function or template next to the email, called where `{{> name}}` stands, with the values in scope there: the item, inside a loop. A partial belongs to the part that defines it, so the HTML and the text each have their own. The newline after `{{/inline}}` renders as nothing | +| `~` in a tag, such as `{{/inline~}}` or `{{~name}}` | The whitespace on that side of the tag is left out | +| `{{! note }}`, `{{!-- note --}}`, `{{log …}}` | Nothing | +| The template's `Subject` / `SubjectPart` | The send's `subject`, rendered with the same conversions and no escaping | +| The template's `Html` / `HtmlPart` | The send's `html`. A template without an HTML part sends `text` only | +| The template's `Text` / `TextPart` | The send's `text`, with no escaping. A template without a text part sends `html` only | + +- Handlebars truth, for `if` and `unless`: `false`, a missing value, `null`, `""`, `0`, and an empty list are false. Everything else is true, including `"0"`, `"false"`, and an empty object. Write that test out in the code, as `template-to-code.md` says. The docs don't say whether a value missing from `TemplateData` that only an `if`, `unless`, `each`, or `with` reads fails the render: treat it as false, and note it in the plan row. +- Any other helper, a `{{> name}}` whose part defines no such inline partial, and the `{{amazonSESUnsubscribeUrl}}` tag, which Amazon SES fills only for a send with `ListManagementOptions`, keep the template on Amazon SES. +- A template belongs to one account and Region: a template of the same name in another Region, or in another AWS account, is another template. Read each one in the Region its site sends in, as `account.md` says. A `TemplateArn` names its Region after `arn:aws:ses:` and its name after `template/`. Two sites that send the same name in two Regions or accounts share one email only when the two contents are the same. A site whose code runs in several deployments, as `account.md` says at its top, sends a template from each one's account and Region: it moves only when the contents are the same in each, and otherwise stays, and the reason is `its template differs between deployments`. +- An Amazon SES template holds no sender, reply-to address, or preview text, so the send takes the site's own `FromEmailAddress` / `Source` and `ReplyToAddresses`, as the "Fields" table says. Its subject always comes from the template: a templated send has no subject of its own. +- Amazon SES hosts no images, so a template's images sit on the person's own hosts, such as an S3 bucket. Keep every URL, and list none as hosted by the provider. +- A template site follows "Open and click tracking" like any other site. Its `ConfigurationSetName` follows the "Fields" table, as for any send. +- `Content.Template.Headers` and `Attachments` keep the site, as the "Fields" table says. `SendBulkTemplatedEmail` and `SendBulkEmail` still stay, as "Calls" says: each sends several emails in one call. + +### Templates made in PostHog + +Only for a template the person asks to make in PostHog, as Step 3 says. The person makes it from the Amazon SES template, makes it transactional, and gives you its template ID. Until then the site stays on Amazon SES. An inline template takes the same path when the person asks for one: read its content from the code for the check. Amazon SES's Handlebars becomes Liquid in the PostHog template. Values move under `variables`, and each name keeps its case: + +| Amazon SES | PostHog | +|---|---| +| `{{name}}` | `{{ variables.name }}` | +| `{{contact.firstName}}` | `{{ variables.contact.firstName }}` | +| `{{#if paid}}…{{else}}…{{/if}}`, when `paid` is a boolean | `{% if variables.paid %}…{% else %}…{% endif %}` | +| `{{#unless paid}}…{{/unless}}`, when `paid` is a boolean | `{% unless variables.paid %}…{% endunless %}` | +| `{{#each items}}{{name}}{{/each}}`, or `{{this.name}}` in the loop, when `items` is a list | `{% for item in variables.items %}{{ item.name }}{% endfor %}` | +| `{{#each tags}}{{this}}{{/each}}`, when `tags` is a list | `{% for tag in variables.tags %}{{ tag }}{% endfor %}` | +| `{{this}}` inside `{{#each items}}` | `{{ item }}` | +| The template's subject (`Subject` or `SubjectPart`) | The PostHog template's subject, with the same conversions | + +- Amazon SES does not escape HTML in values, and PostHog's Liquid does. A value with plain text shows the same text. A value that holds HTML, such as a link or a list the code builds as markup, shows as escaped text in PostHog, so its template stays on Amazon SES. So does a template that uses triple braces, `{{{name}}}`. +- Handlebars treats `0`, an empty string, and an empty list as false, and Liquid treats them as true. Read what the site sends for each `if` and `unless` value. A value that is not always a boolean, `null`, or missing keeps the template on Amazon SES. +- A value the template prints, such as `{{name}}` or a path, that a send can leave out of `TemplateData` or pass as `null`, or a missing step of its path: Amazon SES failed the render and sent nothing, as the table above says, and Liquid renders it as nothing and sends. Check each such value in the code and raise before the send when one is missing, as the render in the code does above, so the email still doesn't go out, but only when the branch that prints it runs for the send's values, under the same `if`, `unless`, and `each` the template has: `{{#if paid}}{{invoiceId}}{{/if}}` with `paid` false prints nothing and raises nothing. Note it in the plan row. A value read only by `if`, `unless`, or `each` follows the bullets on those instead. +- Handlebars `{{#each}}` also walks the values of an object, and Liquid's `for` doesn't. An `each` over a value that is not always a list keeps the template on Amazon SES. +- Inside `{{#each}}`, a bare `{{name}}` reads the current item, as `{{this.name}}` does. It becomes `{{ item.name }}`, never `{{ variables.name }}`. `{{this}}` is the item itself and becomes `{{ item }}`. +- Any other helper, such as `{{#with}}`, `{{lookup}}`, or `{{log}}`, keeps the template on Amazon SES. So does an inline partial (`{{#* inline}}` and `{{> name}}`), `{{@index}}`, `{{@key}}`, and a block inside `{{#each}}` that reads a value outside the loop with `../`. +- `templateData` takes the object `TemplateData` was built from, as above. +- The PostHog template carries its own sender, reply-to address, and subject. A site that sets `FromEmailAddress` / `Source` or `ReplyToAddresses` next to the template moves only when the value is the same in every send, and the person puts it into the PostHog template. Otherwise it stays. +- A template site follows "Open and click tracking" like any other site. + +## Webhooks + +The handlers stop getting events for a site that moves: Amazon SES no longer sends it. PostHog records the events of a moved send on the person it names with `distinctId`, when engagement capture is on: + +| Amazon SES event | PostHog event | +|---|---| +| `Send` | `$workflows_email_sent` | +| `Delivery` | `$workflows_email_delivered` | +| `Bounce` | `$workflows_email_bounced` | +| `Complaint` | `$workflows_email_blocked` | +| `Open` | `$workflows_email_opened` | +| `Click` | `$workflows_email_link_clicked`, only for a template made in PostHog that is not marked sensitive. PostHog treats a send in the code form as sensitive when no template saved in PostHog has its key, which Step 7 makes sure: its links go out as they are, and it counts no clicks | +| `Reject`, `Rendering Failure`, `DeliveryDelay`, `Subscription` | none | + +A bounce to an address on the account-level suppression list comes as a `Bounce` with `bounceSubType` `OnAccountSuppressionList`. PostHog refuses that send instead, with `recipient_suppressed`, when it holds the address as a `bounce`. An address on the list for `COMPLAINT` gets the moved send. + +PostHog's events carry no Amazon SES `mail.messageId` and no `mail.tags`. A site whose events a handler matches by message id or by a tag stays on Amazon SES. A handler fed by an identity's default configuration set gets the events of every send from that identity that names no set: a send that names a set uses that set instead. A handler fed by the identity's SNS notifications (`SetIdentityNotificationTopic`) gets the events of every send from that identity, whatever its set. When either matches by message id or by a tag, every site whose sends it gets stays. A handler that finds the user by the recipient's address, such as `bouncedRecipients[].emailAddress`, keeps no site for that, though the event it acts on can, as below. For every other handler, the report tells the person which events it handles, which PostHog events take their place, and that a webhook destination in PostHog (Data pipelines, then Destinations) can post them to an endpoint. The handler itself keeps working for the sites that stay. Never change it. + +A site whose emails a handler acts on for an event that has no PostHog event in the table, and that its sends can produce, stays: `Reject` and `DeliveryDelay` for any send, `Rendering Failure` only for a templated send, stored or inline, and `Subscription` only for a send with `ListManagementOptions`, which stays anyway. The reason is `its events feed a webhook handler`. + +A site whose clicks Amazon SES tracked, and whose events reach a handler that acts on `Click` events, however it finds its records, moves only to a template made in PostHog that is not marked sensitive, since a send in the code form counts no clicks. Otherwise it stays, and the reason is `its clicks feed a webhook handler`. + +## Suppressions + +PostHog keeps one suppression list per project and refuses every moved send to an address it holds as a `bounce`, the import's `BOUNCE` entries, as `account.md` says under "Map each entry". A `COMPLAINT` entry refuses no moved send. Step 5 imports the entries that apply to any moved site, as `account.md` says under "Map each entry", while Amazon SES checked each send only against its own Region's list, for the reasons its configuration set suppresses. A moved site, counted once per deployment as `account.md` says at its top, therefore has sends refused to addresses Amazon SES delivered to when it sends through another list than another moved site, in another Region or with another AWS account, as `account.md` says at its top, or when its set's reasons leave out `BOUNCE` while another moved site's set suppresses it, such as a set that suppresses only `COMPLAINT`, or a set that turns suppression off. Its code handles that refusal like a send that went out, as "Rewrite" says, so nothing tells the app the email didn't go. + +Work out these sites from the AWS accounts and Regions the sites send with, the sets' suppression settings, and the account's reasons, which `account.md` reads under "Configuration sets" and "Suppression list". A set whose suppression settings can't be read counts as one whose reasons differ. After an earlier run, as `account.md` says under "An earlier import", the lists and reasons its import took count as one more moved site's, since PostHog's list already holds them. The sites earlier runs moved widen too when this run's import adds a list's `BOUNCE` entries that no earlier import took: name them in the question by the code whose PostHog client carries the provider name `amazon-ses`, with the earlier report's "Send call sites" rows whose status starts with "Moved", and as another app's moved sites when another app's migration imported, and ask about the sites this run moves whose entries add it. Ask the person once, for all of them together, whether PostHog may refuse those sends. Yes: they move, and write `suppressions widened: the person agreed` in each one's note under the sites table. No, or no answer: the sites this run would move stay, and the reason is `suppression list wider than Amazon SES's`. Then work it out again for the sites still moving, since the import now takes fewer entries. + +## Auto Validation + +When Amazon SES Auto Validation is on, Amazon SES checks each recipient as it delivers and suppresses those below the person's threshold, reported as a `Bounce` with `bounceSubType` `EmailValidationSuppressed`. PostHog has no such check, and no import can carry it, since it judges each address as it sends. The set's `ValidationOptions` decides for its sends when the set holds one, and the account's `ValidationAttributes` otherwise, for a send with no set too, as `account.md` reads them under "Configuration sets" and "Suppression list". A site whose sends it covers, in any deployment that runs it, whichever project that deployment sends into, stays, and the reason is `Amazon SES Auto Validation checks its recipients`. When the setting that decides can't be read, or the set can't be read, so it is unknown which one decides, ask the person whether Auto Validation is on for those sends, in the Amazon SES console under Email Validation and the set's Auto Validation options; no answer counts as on. + +## Sending turned off + +Amazon SES delivered none of a send's emails when sending was off for it: its configuration set's `SendingOptions.SendingEnabled` is `false`, or, in its Region, the account's `SendingEnabled` is `false`, turned off by the person or paused by AWS, or its `EnforcementStatus` is `SHUTDOWN`. An account whose `ProductionAccessEnabled` is `false` is in the sandbox: Amazon SES sent only to verified addresses and refused every other. PostHog's send has none of these, so a moved site would deliver what Amazon SES didn't. `account.md` reads them under "Configuration sets". A site whose sends one of them covers, in any deployment that runs it, stays, and the reason is `Amazon SES sending is off for it`, or `the Amazon SES account is in the sandbox`, unless the person explicitly agrees that PostHog may deliver those emails. Ask once, naming each such site, the set or account, and the setting. When they agree, the site moves, and its plan row records it, such as `sending off at Amazon SES: the person agreed to deliver through PostHog`. When a setting can't be read, ask the person whether sending is on for the set and the account, in the Amazon SES console under Configuration sets and the Account dashboard, and whether the account has production access; no answer counts as off. + +An IAM policy can refuse sends too: a policy of the role or user a site sends with, or of its sending identity, that limits the action the site calls. It limits it when it allows the action only on some ARNs in `Resource` or only under a `Condition`, or denies it on some ARNs or under a condition, such as a staging role that may mail only the company's domain, or a role that may send only from one address or with one configuration set. The action is `ses:SendEmail` for a v2 send and v1 `SendEmail`, and `ses:SendTemplatedEmail` for v1 `SendTemplatedEmail`, and a wildcard that covers it counts too, such as `ses:Send*`, `ses:*`, or `*`. Amazon SES refused every send outside the limit, and PostHog's project key carries no such limit, so a moved site would deliver what Amazon SES didn't. Read the policies the app's infrastructure code grants its sending roles, such as Terraform, CloudFormation, the AWS CDK, or a `serverless.yml`, and each sending identity's `Policies`, which `account.md` reads under "Configuration sets". Judge the policies together, as AWS does. Among the role's or user's own policies, an Allow limits nothing when another of them grants the same action more widely, such as `AmazonSESFullAccess` or `Resource: *`. A permissions boundary, a session policy, or a service control policy limits on its own, whatever the role's policies grant, and an explicit Deny anywhere limits. A statement for a principal other than the one the app sends with, such as another account in an identity's `Policies`, doesn't apply. Count every condition, whatever its key. Then ask the person once whether a policy outside the code, such as a permissions boundary or a service control policy, limits the app's sends in any way: whom they may mail, which identity, configuration set, template, address, name, or feedback address they may send with, or the network, Region, or tags they need. A site stays when such a limit, in any deployment that runs it, refused or could refuse any of its sends. Check only the values the limit reads: for `Resource`, each ARN the send uses, its identity (its From address or that address's domain, whichever Amazon SES verified), its configuration set (the one it names or the identity's default), and its template; for a condition, the value of each key it tests, such as `ses:Recipients`, `ses:FromAddress`, `ses:FromDisplayName`, `ses:FeedbackAddress`, `aws:ResourceTag/`, or `aws:SourceIp`. One the code builds at run time, one the site leaves unset, one the code doesn't hold, such as a tag, the app's network, or its Region, or a literal the limit doesn't let through could be refused. A site whose every value the limit reads is a literal the limit lets through moves. The reason is `an IAM policy limits its sends`, and the site moves only when the person explicitly agrees that PostHog may deliver those emails, as above. When the person doesn't answer, count a policy outside the code as one that limits every site. + +## Calls that change the suppression list + +`PutSuppressedDestination` adds an address to the account-level suppression list of one Region, with a reason of `BOUNCE` or `COMPLAINT`, `DeleteSuppressedDestination` removes one, and `CreateImportJob` with a `SuppressionListDestination` adds or removes the addresses of a file in Amazon S3. Step 3 and Step 5 say what to do with each such call, such as an unsubscribe handler or a reactivation, in any SDK's spelling, such as `PutSuppressedDestinationCommand` or `delete_suppressed_destination`. They are not sends, and they stay on Amazon SES. + +## Rewrite + +How a site that moves is written with PostHog's SDK. Step 7 says when to use each part, and the rewrite file for the site's language has the client and the send. + +**The PostHog client.** Set the provider name to `amazon-ses` on the PostHog client that Step 7 configures, under the name the rewrite file gives, such as `migratedFrom: 'amazon-ses'` in Node.js or `migrated_from="amazon-ses"` in Python, when the client options take it. + +**The call.** The Amazon SES send becomes one PostHog send. Build its fields from the input the site built: + +- v2 `SendEmail` in any SDK: `FromEmailAddress`, `Destination`, `ReplyToAddresses`, and `Content.Simple` or `Content.Template`. +- v1 `SendEmail` in any SDK: `Source`, `Destination`, `ReplyToAddresses`, and `Message`. v1 `SendTemplatedEmail`: `Source`, `Destination`, `ReplyToAddresses`, `Template` or `TemplateArn`, and `TemplateData`. +- Elixir `ExAws.SES.send_email_v2(destination, content, from, opts)`: the same v2 fields, with `opts[:reply_addresses]` as `ReplyToAddresses`. `ExAws.SES.send_email(dst, msg, from, opts)`: `dst.to`, the `html`, `text`, and `subject` of `build_message`, and `opts[:reply_to]`. `ExAws.SES.send_templated_email(dst, src, template, template_data, opts)`: `dst.to`, `src`, the template's name, the map encoded into `template_data`, and `opts[:reply_to]`. +- A raw HTTP request: the fields of its JSON body or its form parameters. + +**The fields.** + +- `FromEmailAddress` / `Source` becomes `from` as it is, with its name. +- The one entry of `ToAddresses` becomes `to`, as a bare address. When a helper takes a list of recipients and every caller passes one address, change the parameter to one address and update the callers. Never send one email per address in a loop. +- `Subject.Data`, `Body.Html.Data`, and `Body.Text.Data` become `subject`, `html`, and `text`. +- `ConfigurationOverrides.Tracking` is dropped, and the send gets no `captureEngagement` for it, as "Open and click tracking" says. +- A send with rendered content gets a `templateKey`, as Step 7 says. + +**Templates brought into the code.** The site renders the email from the object of its `TemplateData`, as "Templates" says, and sends `templateKey`, the rendered `subject`, `html`, and `text`, and the site's own `from` and `replyTo`. It sends no `templateId` and no `templateData`. An inline template is sent the same way. Remove the encode of `TemplateData` when nothing else uses the string. + +**Templates made in PostHog.** The site sends `templateId` with the template ID the person gave, and `templateData` with the object of the site's `TemplateData`. It sends no `templateKey`, `subject`, `html`, `text`, `from`, or `replyTo`: the template holds them. + +**Errors.** Keep the site failing the same way it did: + +- **Node.js.** `posthog.emails.send` never throws on an API error: it answers `{ data, error }`. Where the Amazon SES send threw into a `catch`, throw from the error so the same `catch` runs: `if (error) throw Object.assign(new Error(error.message), { code: error.code, name: error.name, statusCode: error.statusCode })`. In that `catch`, a read of `err.name` for logging becomes `err.code`, and a read of `err.$metadata.httpStatusCode` becomes `err.statusCode`. A check of `err.name === 'BadRequestException'` or of the status `400` becomes a check of `err.name === 'validation_error'`. +- **Python.** PostHog's SDK raises too. Replace `except ClientError` with the PostHog exception the rewrite file names, `e.response['Error']['Code']` with its code, `e.response['Error']['Message']` with its message, and `e.response['ResponseMetadata']['HTTPStatusCode']` with its status code. A check of the code `BadRequestException` or the status `400` becomes a check of its name `validation_error`. `ClientError` covered only Amazon SES's answers: a network failure raised `EndpointConnectionError` or another `BotoCoreError` past it. PostHog's exception covers both, so raise it again first when its code is `network_error`, unless a later handler of the same `try`, such as `except BotoCoreError` or `except Exception`, took every network failure: then don't raise it, and run that handler's code for `network_error`, as Step 7 says. When the same `try` also guards other AWS calls, keep their `except` as well. +- **Ruby, PHP, and .NET.** PostHog's SDK throws too. Replace the rescued or caught Amazon SES exception (`Aws::SESV2::Errors::ServiceError`, `Aws\SesV2\Exception\SesV2Exception`, `AmazonSimpleEmailServiceV2Exception`, and their v1 names) with the PostHog exception the rewrite file names, and the error code and status code reads with its own. A check of the code `BadRequestException` or the status `400` becomes a check of PostHog's error name `validation_error`. In Ruby and .NET the service exception covered only Amazon SES's answers, and a network failure (`Seahorse::Client::NetworkingError`, `HttpRequestException`) passed it: raise PostHog's exception again first when its code is `network_error`, before the handler logs or does anything else. In .NET, put a `when (e.Code != "network_error")` filter on each `catch` of PostHog's exception that replaces the service exception, so a network failure passes it untouched, as before. The exception is a later handler of the same `try`, such as `rescue Seahorse::Client::NetworkingError` or a `catch (Exception)`, that took every network failure: it can't catch PostHog's exception, so PostHog's catch takes `network_error` with no raise and no filter for it, and runs that handler's code for it, as Step 7 says. When the handler catches the AWS SDK's base exception (`Aws::Errors::ServiceError`, `Aws\Exception\AwsException`, `Amazon.Runtime.AmazonServiceException`), or its `try` also guards other AWS calls, keep that handler for those calls and add PostHog's next to it. +- **A raw HTTP request.** It failed the way its client does. `axios` (unless its `validateStatus` accepts a failed status), `got` and `ky` (unless `throwHttpErrors: false`), Guzzle (unless `'http_errors' => false`), Faraday with `raise_error`, `raise_for_status()`, and `EnsureSuccessStatusCode()` threw on a refused send: the moved site raises PostHog's error the same way, as the language's bullet says, and a read of the client's status in the `catch` reads PostHog's status code. `fetch`, `aws4fetch`'s `AwsClient.fetch`, `requests`, `httpx`, Ruby's `Net::HTTP`, Go's `net/http`, and .NET `HttpClient` without `EnsureSuccessStatusCode()` answered it as a response: a check of that answer's status becomes a check of PostHog's error, and when the code never read the answer, Step 7's rule for an answer the code never read applies. +- **Go, Rust, and Elixir.** The site checked an error value. Check PostHog's error value the way the rewrite file says under "The answer". A plain `if err != nil`, `?`, or `{:error, _}` branch stays as it is. A branch on the status, such as `{:error, {:http_error, 429, _}}`, `HTTPStatusCode()`, or Rust's `raw_response()` status, reads PostHog's status code, except a branch on `400`, Go's `*types.BadRequestException`, or Rust's `SendEmailError::BadRequestException`, which reads PostHog's error name `validation_error`. In Rust, PostHog's `EmailError` has `status()` for the status code, an `Option` such as `Some(429)`, and `kind().as_str()` for the error name. A branch that matched only Amazon SES's answers let a network failure pass to the site's next check: Go's `errors.As` with `smithy.APIError`, `*awshttp.ResponseError`, or a `*types.*` error, Rust's `SdkError::ServiceError`, or Elixir's `{:error, {:http_error, _, _}}`. PostHog returns a network failure as the same error, with the code `network_error`, so give such a branch a check that skips it, such as `emailErr.Code != "network_error"` in Go, `error.code() != "network_error"` in Rust, or a `{:error, %PostHog.Emails.Error{code: "network_error"}}` clause before it in Elixir that takes the path a network failure took before. +- A handler that took only one kind of network failure, such as `except EndpointConnectionError`, which a read timeout passed, keeps the site, as Step 7 says: `moves: no — catches one kind of network failure`. +- A branch on `TooManyRequestsException` becomes a check of PostHog's status code for `429`. So does a branch on the v1 API's `Throttling`: the v1 API answers a send over the sending rate or the daily quota with the code `Throttling` and status `400`, where v2 answers too many requests with `TooManyRequestsException` and `429`. +- PostHog answers a used-up quota with status `429` and the code `quota_limited`, which no retry clears. When a `429` branch tries the send again or waits before it, it also checks for a code other than `quota_limited`, and `quota_limited` takes the path of every other error. Note it in the plan row. +- In a site that sends with the v1 API, a branch on the status `400` also took throttling. It becomes a branch on PostHog's error name `validation_error` or its status code `429`, so a throttled send still takes it. +- A branch on Amazon SES's answer to an invalid send, `BadRequestException` or the status `400`, becomes a branch on PostHog's error name `validation_error`, never on a status code: PostHog's statuses for invalid sends differ from Amazon SES's, and `recipient_suppressed` is a `validation_error` with status `422`, handled first, as below. +- Never widen a catch. When the old code caught only `TooManyRequestsException`, catch PostHog's error, check its status code for `429`, and raise or return every other error as before. When it caught only `BadRequestException`, check PostHog's error name for `validation_error` in the same way, with `e.name`, `#name`, `getErrorName()`, `Name`, or `:name` (`:validation_error` in Elixir): + - **Node.js:** a `catch` that tested `err.name === 'TooManyRequestsException'` tests `err.statusCode === 429` on the error thrown above, and throws every other error again. + - **Python:** `except client.exceptions.TooManyRequestsException` becomes `except EmailError as e:` with `if e.status_code != 429: raise` first. + - **Ruby:** `rescue Aws::SESV2::Errors::TooManyRequestsException` becomes `rescue PostHog::EmailError => e` with `raise unless e.status_code == 429` first. + - **PHP:** a catch that tested `getAwsErrorCode() === 'TooManyRequestsException'` catches `PostHog\EmailError`, tests `getStatusCode() === 429`, and throws every other error again. + - **.NET:** `catch (TooManyRequestsException)` becomes `catch (EmailException e) when (e.StatusCode == 429)`. + - **Go:** `errors.As(err, &throttled)` with `*types.TooManyRequestsException` becomes `errors.As(err, &emailErr) && emailErr.StatusCode == 429`. Every other error goes on to the site's next check as before. + - **Rust:** a match on `SendEmailError::TooManyRequestsException` becomes `Err(error) if error.status() == Some(429)`, and one on `SendEmailError::BadRequestException` becomes `Err(error) if error.kind().as_str() == "validation_error"`. Every other error goes on to the site's next arm as before. +- When other handlers of Amazon SES's errors follow on the same `try`, such as an `except ClientError` or `rescue Aws::SESV2::Errors::ServiceError` after the throttling one, the raise above would skip them: the handlers become one catch of PostHog's error that branches in their order, as Step 7 says, and only an error that none of them took is raised again. + - **Elixir:** `{:error, {:http_error, 429, _}}` becomes `{:error, %PostHog.Emails.Error{status_code: 429}}`. Every other `{:error, _}` goes on as before. +- PostHog refuses a send to a suppressed address with `recipient_suppressed`, where Amazon SES took it and didn't send it. When the site throws, retries, or reports a failure on any error, handle `recipient_suppressed` before that branch the way the site handled a send that went out: log it and go on. A refused send has no message id, where Amazon SES answered one: when the code after the send uses the id for more than a log line, such as returning it to its callers or storing it, the site stays, and the reason is `uses the message id of a suppressed send`, unless its callers already handle a send without an id. Note each site that got this branch in the report. + +**The answer.** The message id of a moved send is PostHog's. Code that logged `MessageId`, or stored it for the app's own use, takes PostHog's message id instead. + +**The Amazon SES client.** Keep the Amazon SES client and its import while any site or other call still uses them. When nothing uses them, remove the import and the client. Keep the package and the AWS credentials for other uses: the AWS SDK package, such as `@aws-sdk/client-sesv2`, `aws-sdk`, `boto3`, `aws-sdk-go-v2`, or `ex_aws`, and the credentials often serve other AWS services in the same app, and `boto3` and `aws-sdk` cover every AWS service in one package. Remove the package only when no code uses it for anything else. The person removes the Amazon SES permissions after the cutover. diff --git a/context/skills/migrate-email/references/amazon-ses/sdk-reference.md b/context/skills/migrate-email/references/amazon-ses/sdk-reference.md new file mode 100644 index 00000000..84d5a9ef --- /dev/null +++ b/context/skills/migrate-email/references/amazon-ses/sdk-reference.md @@ -0,0 +1,274 @@ +# Amazon SES SDK reference + +This file says what Amazon SES looks like in a codebase. It does not describe PostHog. `mapping.md` maps each field to PostHog. + +Amazon SES has two APIs, and both are in use: the SES API v2 (`sesv2`, `SendEmail` with `Content`) and the original SES API (`ses`, `SendEmail` with `Message`). This file calls them v2 and v1. One app can use both. + +## Key + +Amazon SES takes AWS credentials, not an API key. The AWS SDKs read `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`, with the Region in `AWS_REGION`, or `AWS_DEFAULT_REGION` in boto3, from the environment, a shared credentials file, or a role the app runs under. Many apps set no key at all and use a role, so a search for the key's variable can find nothing. These variables often serve other AWS services in the same app too: a place that sets them is not only about email. + +## Packages + +| Language | Package | Send | +|---|---|---| +| Node.js | `@aws-sdk/client-sesv2` | `sesClient.send(new SendEmailCommand({ FromEmailAddress, Destination, Content }))` | +| Node.js | `@aws-sdk/client-ses` | `sesClient.send(new SendEmailCommand({ Source, Destination, Message }))` | +| Node.js | `aws-sdk` (the AWS SDK for JavaScript v2) | `new AWS.SES().sendEmail(params).promise()`, or `new AWS.SESV2().sendEmail(params)` | +| Python | `boto3` | `boto3.client('sesv2').send_email(...)`, or `boto3.client('ses').send_email(...)` | +| Ruby | `aws-sdk-sesv2`, `aws-sdk-ses` | `Aws::SESV2::Client#send_email`, `Aws::SES::Client#send_email` | +| PHP | `aws/aws-sdk-php` | `Aws\SesV2\SesV2Client::sendEmail`, `Aws\Ses\SesClient::sendEmail` | +| Go | `github.com/aws/aws-sdk-go-v2/service/sesv2`, `.../service/ses` | `client.SendEmail(ctx, &sesv2.SendEmailInput{...})` | +| .NET | `AWSSDK.SimpleEmailV2`, `AWSSDK.SimpleEmail` (NuGet) | `AmazonSimpleEmailServiceV2Client.SendEmailAsync`, `AmazonSimpleEmailServiceClient.SendEmailAsync` | +| Rust | `aws-sdk-sesv2` | `client.send_email()...send().await` | +| Java | `software.amazon.awssdk:sesv2`, `software.amazon.awssdk:ses` | `SesV2Client.sendEmail(SendEmailRequest)` | +| Elixir | `ex_aws_ses`, a community package from the `ex-aws` project, not AWS | `ExAws.SES.send_email_v2(...)` or `ExAws.SES.send_email(...)`, then `ExAws.request()` | + +The v1 clients also have `SendTemplatedEmail`, `SendRawEmail`, and `SendBulkTemplatedEmail`, and the v2 clients `SendBulkEmail`: a v2 client sends a template or a MIME message through `SendEmail`. A package that has both clients, such as `aws-sdk`, `boto3`, or `aws/aws-sdk-php`, has all of them. In Node.js they are `SendTemplatedEmailCommand`, `SendRawEmailCommand`, `SendBulkTemplatedEmailCommand`, and `SendBulkEmailCommand`; in Python `send_templated_email`, `send_raw_email`, `send_bulk_templated_email`, and `send_bulk_email`. + +These are not an SES send, and alone they are not a migration: `sns-validator` and `aws/aws-php-sns-message-validator` check the signature of SNS messages, `@aws-sdk/client-sns` and the other AWS service clients talk to other services, and `nodemailer` alone sends through any SMTP server. An SES client that only manages identities, templates, or the suppression list is not a send either. + +## Clients and sends + +**Node.js, v2 API.** `send` answers `{ MessageId }`. It throws on an API error: the error is an `SESv2ServiceException`, `err.name` is the error name, such as `MessageRejected` or `TooManyRequestsException`, `err.message` its text, and `err.$metadata.httpStatusCode` the HTTP status. + +```js +import { SESv2Client, SendEmailCommand } from '@aws-sdk/client-sesv2' + +const ses = new SESv2Client({ region: process.env.AWS_REGION }) + +const { MessageId } = await ses.send(new SendEmailCommand({ + FromEmailAddress: 'Example ', + Destination: { ToAddresses: ['jane@example.com'] }, + Content: { + Simple: { + Subject: { Data: 'Welcome' }, + Body: { Html: { Data: '

Welcome

' }, Text: { Data: 'Welcome' } }, + }, + }, +})) +``` + +**Node.js, v1 API.** The same client shape from `@aws-sdk/client-ses`, with `SESClient` and v1 fields. It throws an `SESServiceException`, with the same `name`, `message`, and `$metadata`. + +```js +import { SESClient, SendEmailCommand } from '@aws-sdk/client-ses' + +await new SESClient({}).send(new SendEmailCommand({ + Source: 'Example ', + Destination: { ToAddresses: ['jane@example.com'] }, + Message: { + Subject: { Data: 'Welcome' }, + Body: { Html: { Data: '

Welcome

' }, Text: { Data: 'Welcome' } }, + }, +})) +``` + +**Node.js, `aws-sdk`.** `new AWS.SES().sendEmail(params)` takes the v1 fields, and `new AWS.SESV2().sendEmail(params)` the v2 fields. `.promise()` answers `{ MessageId }` and rejects on an error, and the callback form `sendEmail(params, (err, data) => ...)` passes the error first. The error has `code` (the error name), `message`, and `statusCode`. + +**Python.** `send_email` answers a dict with `MessageId`. It raises `botocore.exceptions.ClientError` on an API error: `e.response['Error']['Code']` is the error name, `e.response['Error']['Message']` its text, and `e.response['ResponseMetadata']['HTTPStatusCode']` the HTTP status. The client also names each error as a subclass, such as `client.exceptions.MessageRejected` and `client.exceptions.TooManyRequestsException`. + +```python +import boto3 + +ses = boto3.client("sesv2", region_name=os.environ["AWS_REGION"]) +response = ses.send_email( + FromEmailAddress="Example ", + Destination={"ToAddresses": ["jane@example.com"]}, + Content={ + "Simple": { + "Subject": {"Data": "Welcome"}, + "Body": {"Html": {"Data": "

Welcome

"}, "Text": {"Data": "Welcome"}}, + } + }, +) +message_id = response["MessageId"] +``` + +The v1 client is `boto3.client("ses")`, with `send_email(Source=, Destination=, Message={"Subject": {...}, "Body": {...}})`. + +**Ruby.** `send_email` answers a response whose `message_id` is the message id. It raises an `Aws::SESV2::Errors::ServiceError` on an API error, such as `Aws::SESV2::Errors::MessageRejected`, with `e.code`, `e.message`, and `e.context.http_response.status_code`. + +```ruby +require 'aws-sdk-sesv2' + +ses = Aws::SESV2::Client.new(region: ENV['AWS_REGION']) +resp = ses.send_email( + from_email_address: 'Example ', + destination: { to_addresses: ['jane@example.com'] }, + content: { simple: { + subject: { data: 'Welcome' }, + body: { html: { data: '

Welcome

' }, text: { data: 'Welcome' } } + } } +) +``` + +The v1 client is `Aws::SES::Client`, with `send_email(source:, destination:, message:)` and `Aws::SES::Errors::ServiceError`. + +**PHP.** `sendEmail` answers an `Aws\Result`, and `$result['MessageId']` is the message id. It throws `Aws\SesV2\Exception\SesV2Exception` on an API error, or `Aws\Ses\Exception\SesException` for the v1 client. Both are `Aws\Exception\AwsException`, with `getAwsErrorCode()`, `getAwsErrorMessage()`, and `getStatusCode()`. + +```php +use Aws\SesV2\SesV2Client; + +$ses = new SesV2Client(['region' => getenv('AWS_REGION'), 'version' => 'latest']); +$result = $ses->sendEmail([ + 'FromEmailAddress' => 'Example ', + 'Destination' => ['ToAddresses' => ['jane@example.com']], + 'Content' => ['Simple' => [ + 'Subject' => ['Data' => 'Welcome'], + 'Body' => ['Html' => ['Data' => '

Welcome

'], 'Text' => ['Data' => 'Welcome']], + ]], +]); +``` + +**Go.** `SendEmail` answers `(*sesv2.SendEmailOutput, error)`, and `out.MessageId` is a `*string`. It returns an error value, never panics: `errors.As` with `smithy.APIError` (`github.com/aws/smithy-go`) reads `ErrorCode()` and `ErrorMessage()`, a typed error such as `*types.MessageRejected` matches one error, and `*awshttp.ResponseError` (`github.com/aws/aws-sdk-go-v2/aws/transport/http`) has `HTTPStatusCode()`. + +```go +cfg, err := config.LoadDefaultConfig(ctx) +client := sesv2.NewFromConfig(cfg) +out, err := client.SendEmail(ctx, &sesv2.SendEmailInput{ + FromEmailAddress: aws.String("Example "), + Destination: &types.Destination{ToAddresses: []string{"jane@example.com"}}, + Content: &types.EmailContent{Simple: &types.Message{ + Subject: &types.Content{Data: aws.String("Welcome")}, + Body: &types.Body{ + Html: &types.Content{Data: aws.String("

Welcome

")}, + Text: &types.Content{Data: aws.String("Welcome")}, + }, + }}, +}) +``` + +The v1 client is `ses.NewFromConfig(cfg)` from `.../service/ses`, with `ses.SendEmailInput{Source, Destination, Message}`. + +**.NET.** `SendEmailAsync` answers a `SendEmailResponse` whose `MessageId` is the message id. It throws on an API error: an `AmazonSimpleEmailServiceV2Exception`, such as `MessageRejectedException`, with `ErrorCode`, `Message`, and `StatusCode`. The namespaces are `Amazon.SimpleEmailV2` and `Amazon.SimpleEmailV2.Model`. + +```csharp +var ses = new AmazonSimpleEmailServiceV2Client(); +var response = await ses.SendEmailAsync(new SendEmailRequest +{ + FromEmailAddress = "Example ", + Destination = new Destination { ToAddresses = new List { "jane@example.com" } }, + Content = new EmailContent + { + Simple = new Message + { + Subject = new Content { Data = "Welcome" }, + Body = new Body { Html = new Content { Data = "

Welcome

" }, Text = new Content { Data = "Welcome" } }, + }, + }, +}); +``` + +The v1 client is `AmazonSimpleEmailServiceClient` in `Amazon.SimpleEmail`, with `SendEmailRequest { Source, Destination, Message }` and `AmazonSimpleEmailServiceException`. + +**Rust.** `send()` answers `Result>`, and `out.message_id()` is the message id. `err.into_service_error()` gives a `SendEmailError`, with one variant per error, such as `SendEmailError::MessageRejected`, and `code()`. + +```rust +let config = aws_config::load_from_env().await; +let client = aws_sdk_sesv2::Client::new(&config); +let out = client + .send_email() + .from_email_address("Example ") + .destination(Destination::builder().to_addresses("jane@example.com").build()) + .content(EmailContent::builder().simple( + Message::builder() + .subject(Content::builder().data("Welcome").build()?) + .body(Body::builder().html(Content::builder().data("

Welcome

").build()?).build()) + .build(), + ).build()) + .send() + .await?; +``` + +**Elixir.** `ExAws.request()` answers `{:ok, %{body: body}}`, or `{:error, {:http_error, status, body}}` on an API error. + +```elixir +ExAws.SES.send_email_v2( + %{ToAddresses: ["jane@example.com"]}, + %{Simple: %{Subject: %{Data: "Welcome"}, Body: %{Html: %{Data: "

Welcome

"}, Text: %{Data: "Welcome"}}}}, + "Example " +) +|> ExAws.request() +``` + +- `send_email_v2(destination, content, from, opts)` takes the v2 fields, and its options `:reply_addresses`, `:tags`, `:configuration_set_name`, `:list_management`, `:feedback_forwarding_address`, and `:from_arn`. +- `send_email(%{to: [...], cc: [...], bcc: [...]}, ExAws.SES.build_message(html, text, subject), from, opts)` takes the v1 fields, and its options `:reply_to`, `:tags`, `:configuration_set_name`, `:return_path`, `:source_arn`, and `:return_path_arn`. +- `send_templated_email`, `send_raw_email`, and `send_bulk_templated_email` are the v1 template, raw, and bulk sends. + +**Throttling.** The v2 API answers too many requests with `TooManyRequestsException` and status `429`. The v1 API answers a send over the sending rate or the daily quota with the code `Throttling` and status `400`, with the message `Maximum sending rate exceeded` or `Daily message quota exceeded`. + +**HTTP.** The v2 send is `POST https://email..amazonaws.com/v2/email/outbound-emails` with a JSON body of the v2 fields, and the v1 send is a `POST` to `https://email..amazonaws.com/` with `Action=SendEmail` and the v1 fields as form parameters. Both are signed with AWS Signature Version 4, for the service `ses`. A success answers the `MessageId`. Look for `email.` and `amazonaws.com` in `fetch`, `axios`, `requests`, `httpx`, `net/http`, and `curl` calls. + +**Regions.** Each client sends to the host of one Region, `email..amazonaws.com`, and the code picks the Region by a setting, not by a URL. The signal is the client's Region setting: `region` in the Node.js clients, `region_name` in `boto3.client`, `region:` in `Aws::SESV2::Client.new`, `'region'` in the PHP client's options, `config.WithRegion` in Go's `config.LoadDefaultConfig`, a `RegionEndpoint` passed to the .NET client, `.region(...)` on Rust's `aws_config` loader, and `region:` in `ExAws.request` or in `config :ex_aws`. Without one, the SDKs read `AWS_REGION` or the shared config file. boto3 reads `AWS_DEFAULT_REGION`, not `AWS_REGION`. A client can also name its own host with an endpoint setting: `endpoint` in the Node.js clients, `endpoint_url` in `boto3.client`, `config.WithBaseEndpoint` in Go, and `.endpoint_url(...)` in Rust. `account.md` says which host its requests use for each. + +**SMTP.** Amazon SES takes mail over SMTP at `email-smtp..amazonaws.com`, on port 25, 587, or 2587 with STARTTLS, or 465 or 2465 with TLS, with SMTP credentials. Look for that host in Nodemailer transports, `EMAIL_HOST`, `MAIL_HOST`, or `SMTP_HOST` settings, and framework mail config. + +**Nodemailer's SES transport.** `nodemailer.createTransport({ SES: { sesClient, SendEmailCommand } })` sends every `transporter.sendMail(...)` through `@aws-sdk/client-sesv2`. Each `sendMail` that uses it is a site. + +**Frameworks.** Mailers that send every framework mail through Amazon SES: `django-ses` (`EMAIL_BACKEND = 'django_ses.SESBackend'`) and `django-anymail` with its Amazon SES backend in Django, `aws-actionmailer-ses` (`delivery_method = :ses` or `:ses_v2`) in Rails, the `ses` and `ses-v2` mailers (`MAIL_MAILER=ses`) in Laravel, `symfony/amazon-mailer` in Symfony, and `Swoosh.Adapters.AmazonSES`, `Swoosh.Adapters.ExAwsAmazonSES`, and `bamboo_ses` in Elixir. Each mailable or mailer that sends is a site. + +## Send fields + +The v2 names, with the v1 names where they differ. The SDKs use the same names in their own case, such as `from_email_address` in Ruby and `FromEmailAddress` in Python. + +| v2 field | v1 field | Value | +|---|---|---| +| `FromEmailAddress` | `Source` | An address, or `Name
` | +| `Destination.ToAddresses` | `Destination.ToAddresses` | A list of addresses. Each can be `Name
` | +| `Destination.CcAddresses`, `BccAddresses` | the same | Same as `ToAddresses`. One send takes at most 50 recipients across the three | +| `ReplyToAddresses` | `ReplyToAddresses` | A list of addresses | +| `Content.Simple.Subject` | `Message.Subject` | `{ Data, Charset }` | +| `Content.Simple.Body.Html`, `Body.Text` | `Message.Body.Html`, `Body.Text` | `{ Data, Charset }` | +| `Content.Simple.Headers`, `Content.Template.Headers` | none | A list of `{ Name, Value }`, at most 15 | +| `Content.Simple.Attachments`, `Content.Template.Attachments` | none | A list of `{ FileName, RawContent, ContentType, ContentId, ContentDisposition, ContentDescription, ContentTransferEncoding }`. `RawContent` is the bytes, and base64 over raw HTTP and in Elixir's `ex_aws_ses`, which sends `Content` as JSON as it is. `ContentDisposition` is `ATTACHMENT` or `INLINE`, and `ContentId` names an `INLINE` file for the HTML | +| `Content.Template.TemplateName`, `TemplateArn` | `Template`, `TemplateArn` (`SendTemplatedEmail`) | A template stored at Amazon SES | +| `Content.Template.TemplateData` | `TemplateData` | A JSON string of the template's values | +| `Content.Template.TemplateContent` | none | `{ Subject, Html, Text }`: an inline template in the code | +| `Content.Raw.Data` | `RawMessage.Data` (`SendRawEmail`) | A whole MIME message | +| `EmailTags` | `Tags` | A list of `{ Name, Value }`, sent with the send's events | +| `ConfigurationSetName` | `ConfigurationSetName` | A configuration set: its event destinations, tracking, sending pool, and suppression settings. A send that names none uses its identity's default set, when it has one | +| `ConfigurationOverrides.Tracking` | none | `{ OpenTrackingEnabled, ClickTrackingEnabled }`, each `ENABLED` or `DISABLED`, for this send only | +| `ListManagementOptions` | none | `{ ContactListName, TopicName }`: a contact list and topic for unsubscribes | +| `FeedbackForwardingEmailAddress` | `ReturnPath` | The address that gets bounce and complaint mail when feedback forwarding is on | +| `FromEmailAddressIdentityArn`, `FeedbackForwardingEmailAddressIdentityArn` | `SourceArn`, `ReturnPathArn`, `FromArn` | Sending authorization: an identity another AWS account owns | +| `EndpointId` | none | A multi-region endpoint | +| `TenantName` | none | A tenant, with its own identities, configuration sets, templates, and suppression list | + +One send takes at most 10 MB. + +## Templates stored at Amazon SES + +A v2 send with `Content.Template.TemplateName` or `TemplateArn`, or a v1 `SendTemplatedEmail` with `Template`, renders a template stored in the person's Amazon SES account, with the values in `TemplateData`. Its content is not in the code. + +- A stored template holds a subject, an HTML part, and a text part: `TemplateContent.{Subject, Html, Text}` in v2, and `SubjectPart`, `HtmlPart`, and `TextPart` in v1. +- A stored template uses Handlebars: `{{name}}`, nested paths such as `{{contact.firstName}}`, `{{#if}}…{{else}}…{{/if}}`, `{{#each}}…{{/each}}`, inline partials with `{{#* inline "name"}}…{{/inline}}` and `{{> name}}`, and the other Handlebars built-in helpers. +- Amazon SES does not escape HTML in values when it renders the HTML part. A value inserts as it is. +- An inline template, `Content.Template.TemplateContent`, has its content in the code, and takes only simple `{{name}}` substitutions. +- Amazon SES accepts a templated send whose values don't render, answers a `MessageId`, and doesn't send it. It reports a `Rendering Failure` event instead. + +## Status reads + +`GetMessageInsights` (`GET https://email..amazonaws.com/v2/email/insights/{MessageId}/`) reads one message's events from the Virtual Deliverability Manager. Code that calls it reads Amazon SES's state. + +## Webhooks + +Amazon SES reports what happens to a send in two ways. Both post to Amazon SNS, or to another AWS service, and an app reads them from there: + +- **Event publishing.** A configuration set has event destinations: Amazon SNS, Amazon EventBridge, Amazon Data Firehose, Amazon CloudWatch, or Amazon Pinpoint. Each event has `eventType`, one of `Send`, `Delivery`, `Bounce`, `Complaint`, `Reject`, `Open`, `Click`, `Rendering Failure`, `DeliveryDelay`, and `Subscription`. +- **Identity notifications.** An identity can post its bounces, complaints, and deliveries to an SNS topic. Each notification has `notificationType`, one of `Bounce`, `Complaint`, and `Delivery`. + +Both carry a `mail` object with `messageId` (the `MessageId` the send answered), `timestamp`, `source`, `destination` (the recipients), and, for events, `tags` (the send's `EmailTags`). A `bounce` object has `bounceType` (`Permanent`, `Transient`, or `Undetermined`), `bounceSubType`, and `bouncedRecipients[].emailAddress`. A `complaint` object has `complainedRecipients[].emailAddress`. A `delivery` object has `recipients`. + +An app handler is usually an HTTP route that an SNS topic posts to. Look for: + +- The `x-amz-sns-message-type` header, with `SubscriptionConfirmation`, `Notification`, or `UnsubscribeConfirmation`, or a body `Type` with the same values. +- A visit to the body's `SubscribeURL`, which confirms the subscription. +- `JSON.parse(body.Message)` or `json.loads(body["Message"])`, then a read of `eventType` or `notificationType`, `mail.messageId`, `mail.tags`, `bouncedRecipients`, or `complainedRecipients`. +- A signature check of the body's `Signature`, `SignatureVersion`, and `SigningCertURL`, often with `sns-validator` or `aws/aws-php-sns-message-validator`. + +A queue consumer or a function in the repository that reads the same JSON from SNS, SQS, or EventBridge is a handler too. + +## Other APIs + +Identities and DKIM (`CreateEmailIdentity`, `GetEmailIdentity`, `VerifyEmailIdentity`), configuration sets and event destinations, templates (`CreateEmailTemplate`, `GetEmailTemplate`, `CreateTemplate`, `GetTemplate`), contact lists and contacts, the suppression list (`ListSuppressedDestinations`, `PutSuppressedDestination`, `DeleteSuppressedDestination`, `CreateImportJob`, which `mapping.md` names under "Calls that change the suppression list"), account and quota reads (`GetAccount`, `GetSendQuota`, `GetSendStatistics`), receiving (receipt rules), and custom verification emails (`SendCustomVerificationEmail`, which verifies an address for the SES account) are not transactional sends. diff --git a/scripts/lib/tests/migrate-email-skill.test.js b/scripts/lib/tests/migrate-email-skill.test.js index 0cff7a9c..76abee5d 100644 --- a/scripts/lib/tests/migrate-email-skill.test.js +++ b/scripts/lib/tests/migrate-email-skill.test.js @@ -11,7 +11,7 @@ const DOCS_PAGES = { 'https://posthog.com/docs/workflows/transactional-emails/api.md': '# Send transactional emails with the API\n', }; const REFERENCE_PATH = /references\/[A-Za-z0-9_./-]+\.md/g; -const PROVIDERS = ['resend']; +const PROVIDERS = ['resend', 'amazon-ses']; const PROVIDER_FILES = ['sdk-reference.md', 'mapping.md', 'account.md']; const SDKS = ['node', 'python', 'go', 'ruby', 'php', 'dotnet', 'rust', 'elixir'];