Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 0 additions & 76 deletions .github/workflows/deploy.yml

This file was deleted.

360 changes: 180 additions & 180 deletions README.md

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions scripts/assemble_site.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
# Lays out the site that gets served, from what scripts/gen_leaderboard_data.py generated:
# site/public/ for GitHub Pages (deploy.yml), and the same tree without the run logs for
# the genhttp.dev lambda (publish_genhttp.py), which sends the site's log links on to the
# logs in this repository instead.
# without the run logs for the genhttp.dev lambda (publish_genhttp.py), which sends the site's
# log links on to the logs in this repository instead, and with them in site/public/ to look
# at the whole site locally (python3 -m http.server -d site/public).
#
# scripts/assemble_site.sh [--no-logs] [out-dir] out-dir defaults to site/public
set -euo pipefail
Expand Down
14 changes: 7 additions & 7 deletions scripts/gen_leaderboard_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -653,7 +653,7 @@ def build_rounds():
# without this every doc collapses into the single "/" URL. Content is the exact
# HTML build_docs() already produces; only the link scheme differs (#doc=x -> /docs/x/).

SITE = "https://www.http-arena.com"
SITE = "https://http-arena.com"
GEN = ROOT / "site" / "generated"
DOCS_OUT = GEN / "docs"

Expand Down Expand Up @@ -3353,7 +3353,7 @@ def build_compare_pages(profiles, results, lang_rows, members_by_lang, compare_l
_og_card(dest / "og.png", "Head to head · " + SCOPE_NAME[DEFAULT_SCOPE],
lang + " web frameworks, compared",
[(r[0], "%.0f" % r[3]) for r in rows[:5]],
"%d entries · %s · www.http-arena.com" % (len(members), round_name))
"%d entries · %s · http-arena.com" % (len(members), round_name))
cards += 1
written.append((lang, len(members)))

Expand All @@ -3374,7 +3374,7 @@ def build_compare_pages(profiles, results, lang_rows, members_by_lang, compare_l
_og_card(dest / "og.png", "Head to head · every language",
"Compare any two entries, whatever they are written in",
[(r[0], "%.0f" % r[3]) for r in all_rows[:5]],
"%d entries · %s · www.http-arena.com" % (len(all_members), round_name))
"%d entries · %s · http-arena.com" % (len(all_members), round_name))
cards += 1
written.insert(0, (None, len(all_members)))

Expand All @@ -3383,7 +3383,7 @@ def build_compare_pages(profiles, results, lang_rows, members_by_lang, compare_l
"Compare two web frameworks, profile by profile",
[(lang or "Every language", "%d entries" % n) for lang, n in
sorted(written, key=lambda x: -x[1])[:5]],
"%d languages · %s · www.http-arena.com" % (len(written), round_name))
"%d languages · %s · http-arena.com" % (len(written), round_name))
cards += 1
(COMPARE_OUT / "index.html").write_text(
_compare_index_page(written, updated, "/compare/og.png" if with_og else ""),
Expand Down Expand Up @@ -3483,7 +3483,7 @@ def build_fw_pages(profiles, results, meta, fw_lang, badge_index, achievements,
[(SCOPE_NAME[s], "#%d of %d" % (rank, field))
for s, rank, field, _, _link in ranks],
" · ".join(x for x in [lang, TYPE_LABEL.get(kind, kind),
round_name, "www.http-arena.com"] if x),
round_name, "http-arena.com"] if x),
blurb=m.get("desc", ""))
cards += 1
entries.append((fw, lang, kind))
Expand Down Expand Up @@ -3518,14 +3518,14 @@ def build_og_images(content, trails, rows, fw_lang, round_name):
"Composite ranking · " + SCOPE_NAME[DEFAULT_SCOPE],
"Which web framework is actually fastest?",
top,
f"{len(rows)} entries · {round_name} · www.http-arena.com")
f"{len(rows)} entries · {round_name} · http-arena.com")
written = 1

for did, d in content.items():
crumb = " › ".join(t for t, _ in trails.get(did, [])[:-1]) or "Knowledge Base"
_og_card(DOCS_OUT / did / "og.png" if did else DOCS_OUT / "og.png",
crumb, d["t"] or "Knowledge Base", [],
"HttpArena Knowledge Base · www.http-arena.com",
"HttpArena Knowledge Base · http-arena.com",
blurb=d.get("d") or _meta_desc(d["html"]))
written += 1
return "/og.png", written
Expand Down
2 changes: 1 addition & 1 deletion scripts/validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,7 @@ fi

# ───── Helpers ─────

DOCS_BASE="https://www.http-arena.com/#doc=test-profiles"
DOCS_BASE="https://http-arena.com/#doc=test-profiles"

# Every parameter the profiles below send is drawn fresh per run. The suite used
# to ask for a fixed set - /json/12?m=3, /json/50?m=1, min=10&max=50&limit=50 -
Expand Down
14 changes: 7 additions & 7 deletions site/content/docs/add-framework/badges.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ itself each time the leaderboard is republished. You paste it once.
## Paste it

```md
[![HTTP Arena](https://img.shields.io/endpoint?url=https://www.http-arena.com/badge/actix/h1.json)](https://www.http-arena.com/#type=emerging,flagship&tuned=0)
[![HTTP Arena](https://img.shields.io/endpoint?url=https://http-arena.com/badge/actix/h1.json)](https://http-arena.com/#type=emerging,flagship&tuned=0)
```

Replace `actix` with your entry and `h1` with the family you want. The exact
line for your framework, already assembled and deep-linked to the right view of
the board, is in [`/badge/index.json`](https://www.http-arena.com/badge/index.json)
the board, is in [`/badge/index.json`](https://http-arena.com/badge/index.json)
under `scopes.<family>.default.markdown`.

## Families
Expand All @@ -44,13 +44,13 @@ Optional, and a second badge rather than a replacement. Append `-<language>` to
the family and the badge reads **`#1 of 6 (Rust)`**:

```md
[![HTTP Arena](https://img.shields.io/endpoint?url=https://www.http-arena.com/badge/actix/h1-rust.json)](https://www.http-arena.com/#type=emerging,flagship&tuned=0&lang=Rust)
[![HTTP Arena](https://img.shields.io/endpoint?url=https://http-arena.com/badge/actix/h1-rust.json)](https://http-arena.com/#type=emerging,flagship&tuned=0&lang=Rust)
```

The language segment is the board's own language name, lowercased, with `#`
spelled out as `sharp` and `++` as `pp`: `C#` → `csharp`, `C++` → `cpp`,
`TS` → `ts`, `JS` → `js`. Your ready-made line is in
[`/badge/index.json`](https://www.http-arena.com/badge/index.json) under
[`/badge/index.json`](https://http-arena.com/badge/index.json) under
`scopes.<family>.byLanguage.markdown`.

## Counting tuned entries in
Expand All @@ -61,7 +61,7 @@ someone else's hand-tuned build. Append `-with-tuned` to count tuned entries
as well:

```md
[![HTTP Arena](https://img.shields.io/endpoint?url=https://www.http-arena.com/badge/actix/h1-with-tuned.json)](https://www.http-arena.com/#type=emerging,flagship&tuned=1)
[![HTTP Arena](https://img.shields.io/endpoint?url=https://http-arena.com/badge/actix/h1-with-tuned.json)](https://http-arena.com/#type=emerging,flagship&tuned=1)
```

The suffixes combine, so the full set for one family is:
Expand All @@ -73,7 +73,7 @@ The suffixes combine, so the full set for one family is:
| `h1-rust.json` | standard, one language |
| `h1-rust-with-tuned.json` | standard + tuned, one language |

Each is in [`/badge/index.json`](https://www.http-arena.com/badge/index.json)
Each is in [`/badge/index.json`](https://http-arena.com/badge/index.json)
under `scopes.<family>`, keyed `default`, `withTuned`, `byLanguage` and
`byLanguageWithTuned`.

Expand Down Expand Up @@ -102,7 +102,7 @@ type swatch beside every framework name:
| 🟧 | `experimental` | other experimental entries |
| 🟥 | `engine` | other engines, on the profiles engines are scored on |

The tier is also in [`/badge/index.json`](https://www.http-arena.com/badge/index.json)
The tier is also in [`/badge/index.json`](https://http-arena.com/badge/index.json)
as `type`, next to each entry's ranks.

## What the number means
Expand Down
2 changes: 1 addition & 1 deletion site/content/docs/load-generators/ws/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ gcannon reports WebSocket results with the same layout as HTTP requests, except
WS frames: 2400000
```

The parser (`gcannon_parse ws-echo`) records `frames received` as the `status_2xx` equivalent and divides by the measured duration to produce the headline RPS number shown on the [WebSocket leaderboard](https://www.http-arena.com/#scope=ws). One echo round-trip counts as one unit - the frames-received count from the client side, not frames-sent, because the metric is "how many echoes the framework completed," not "how many messages the benchmarker pushed into the socket."
The parser (`gcannon_parse ws-echo`) records `frames received` as the `status_2xx` equivalent and divides by the measured duration to produce the headline RPS number shown on the [WebSocket leaderboard](https://http-arena.com/#scope=ws). One echo round-trip counts as one unit - the frames-received count from the client side, not frames-sent, because the metric is "how many echoes the framework completed," not "how many messages the benchmarker pushed into the socket."

## Why not a dedicated WebSocket tool

Expand Down
4 changes: 2 additions & 2 deletions site/content/docs/test-profiles/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ The endpoints are split across two files:

| Spec | Profiles covered | |
|------|-----------------|---|
| [openapi.json](/openapi.json) | All HTTP/1.1, HTTP/2, HTTP/3, gRPC, and WebSocket profiles | [Browse](https://redocly.github.io/redoc/?url=https://www.http-arena.com/openapi.json) |
| [openapi-gateway.json](/openapi-gateway.json) | Gateway profiles (production-stack) | [Browse](https://redocly.github.io/redoc/?url=https://www.http-arena.com/openapi-gateway.json) |
| [openapi.json](/openapi.json) | All HTTP/1.1, HTTP/2, HTTP/3, gRPC, and WebSocket profiles | [Browse](https://redocly.github.io/redoc/?url=https://http-arena.com/openapi.json) |
| [openapi-gateway.json](/openapi-gateway.json) | Gateway profiles (production-stack) | [Browse](https://redocly.github.io/redoc/?url=https://http-arena.com/openapi-gateway.json) |

## Benchmark parameters

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ On the JSON Compressed tab of the main leaderboard, the Score column is computed

### Composite score column

On the [composite leaderboard](https://www.http-arena.com/), the JSON Compressed column follows a different aggregation path:
On the [composite leaderboard](https://http-arena.com/), the JSON Compressed column follows a different aggregation path:

1. Average `rps` and `bandwidth` across the 3 conn counts (arithmetic mean, equal weight).
2. Compute `myBpr = avgBw / avgRps` from those averages.
Expand Down
22 changes: 13 additions & 9 deletions site/lambda/Site.cs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// The site: what scripts/gen_leaderboard_data.py generated and scripts/assemble_site.sh laid out
// at a commit of main - the tree GitHub Pages serves, without the logs. It is data, like the
// at a commit of main - the tree GitHub Pages served, without the logs. It is data, like the
// results beside it: the HttpArena repository sends it through api/ingest/site/ when its publish
// workflow is run, no version is saved for it, and a deploy leaves it as it is.
//
Expand All @@ -11,8 +11,9 @@
// changes finds its file.
//
// A folder answers with its index.html (docs/scoring/ is docs/scoring/index.html), a folder
// asked for without its slash is sent on to it as GitHub Pages does, and a path that is nothing
// answers 404 with the site's own 404.html.
// asked for without its slash is sent on to it as GitHub Pages did, and a path that is nothing
// answers 404 with the site's own 404.html. Any origin may read it, as on Pages: the docs open
// openapi.json in Redoc, on an origin of its own.

using System.Security.Cryptography;

Expand Down Expand Up @@ -291,7 +292,7 @@ private static async Task<byte[]> ReadAsync(Stream body)
}
}

/// <summary>Answers a path of the site the way GitHub Pages does.</summary>
/// <summary>Answers a path of the site the way GitHub Pages did.</summary>
public sealed class SiteHandler(SiteStore site) : IHandler
{
private static readonly ContentType Html = new("text/html; charset=utf-8");
Expand All @@ -304,7 +305,7 @@ public async ValueTask<IResponse> HandleAsync(IRequest request)
{
if (!request.HasType(RequestMethod.Get, RequestMethod.Head))
{
return request.Respond().Status(ResponseStatus.MethodNotAllowed).Header("Allow", "GET, HEAD").Build();
return Respond(request).Status(ResponseStatus.MethodNotAllowed).Header("Allow", "GET, HEAD").Build();
}

var target = request.Header.Target;
Expand All @@ -315,12 +316,12 @@ public async ValueTask<IResponse> HandleAsync(IRequest request)

if (await site.OpenAsync(name) is (var resource, var type))
{
return request.Respond().Content(resource, type).Build();
return Respond(request).Content(resource, type).Build();
}

if (!site.Published)
{
return request.Respond()
return Respond(request)
.Status(ResponseStatus.ServiceUnavailable)
.Content("The site was not published here yet: the HttpArena repository sends it with scripts/publish_genhttp.py.", Text)
.Build();
Expand All @@ -329,14 +330,17 @@ public async ValueTask<IResponse> HandleAsync(IRequest request)
if (path.Length > 0 && !target.HasTrailingSlash && site.Has(path + "/index.html"))
{
// relative, as every address here is: "scoring/" from docs/scoring
return request.Respond()
return Respond(request)
.Status(ResponseStatus.MovedPermanently)
.Header("Location", path[(path.LastIndexOf('/') + 1)..] + "/")
.Build();
}

var notFound = request.Respond().Status(ResponseStatus.NotFound);
var notFound = Respond(request).Status(ResponseStatus.NotFound);

return (await site.OpenAsync("404.html") is (var page, _) ? notFound.Content(page, Html) : notFound.Content("Not found.", Text)).Build();
}

/// <summary>Every answer of the site, readable from any origin.</summary>
private static IResponseBuilder Respond(IRequest request) => request.Respond().Header("Access-Control-Allow-Origin", "*");
}
10 changes: 6 additions & 4 deletions site/lambda/docs/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ the editor, or pushed to this lambda's own git repository, is replaced by the ne
## The site is data, not a version

The site is what `scripts/gen_leaderboard_data.py` generated and `scripts/assemble_site.sh
--no-logs` laid out: the same tree GitHub Pages serves, without the logs. Nearly every page of it
--no-logs` laid out: the same tree GitHub Pages served, without the logs. Nearly every page of it
is generated from the results, so it changes whenever they do, and a version for every such
change is what the owner did not want: a version is the program. So the site is sent like the results,
through `api/ingest/site/`, and kept in the workspace; a deploy leaves it alone, and a
Expand All @@ -27,10 +27,12 @@ of a ~1,770-file site are sent the first time and only what changed after that.
neither the new paths nor the ones before them is deleted then, so a page in the middle of being
sent while the site changes still finds its file.

`Site.cs` serves it the way GitHub Pages does: a folder answers with its `index.html`, a folder
`Site.cs` serves it the way GitHub Pages did: a folder answers with its `index.html`, a folder
asked for without its slash is sent on to it with a 301, and a path that is nothing answers 404
with the site's own `404.html` - never the `index.html` of a folder above it. Types come from the
names, and ETags from the files, so a browser that has a page is told it is unchanged.
names, and ETags from the files, so a browser that has a page is told it is unchanged. Every
answer carries `Access-Control-Allow-Origin: *`, as on Pages: the docs' "Browse" links open
`openapi.json` in Redoc, which reads it from an origin of its own.

The pages link absolutely (`/docs/...`, `/data.js`): they are generated for the root of a domain.
That works at the lambda's own address and on a domain of its own, and not in a feature's preview
Expand All @@ -54,7 +56,7 @@ are what everything that reads results already understands. They are built per r

The container log of every run is kept where `benchmark.sh` writes it, in the repository under
`site/static/logs/`, as the owner wants, and this lambda keeps no copy. The board links a log at
`logs/{profile}/{conns}/{framework dir}.log`, as it does on GitHub Pages; here that answers 302 to
`logs/{profile}/{conns}/{framework dir}.log`, as it did on GitHub Pages; here that answers 302 to
the same file on `raw.githubusercontent.com`, as it is on main. The redirect is temporary so that
browsers don't remember it, and the links work wherever the board is served from. The lambda
kept gzipped copies until `V3__Drop_logs.sql` dropped their table.
Expand Down
2 changes: 1 addition & 1 deletion site/lambda/tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- The site: `/` is the board, a folder answers with its `index.html` (`docs/scoring/`), a folder
without its slash is sent on to it (`docs` answers 301 to `docs/`), and a path that is nothing
answers 404 with the site's `404.html`. Every page answers exactly what `assemble_site.sh` laid
out at the commit `api/site` names.
out at the commit `api/site` names, with `Access-Control-Allow-Origin: *`.
- Publishing the same site twice sends nothing the second time; content no longer named by the
site or the one before it is deleted.
- `api/frameworks`, `api/results/{slug}` and `api/host` answer exactly what the repository's
Expand Down
Loading
Loading