Skip to content

zulip: Add fetch_upload() for downloading uploaded files. - #948

Open
harshvardhan178251 wants to merge 3 commits into
zulip:mainfrom
harshvardhan178251:fix-628-fetch-upload
Open

harshvardhan178251 wants to merge 3 commits into
zulip:mainfrom
harshvardhan178251:fix-628-fetch-upload

Conversation

@harshvardhan178251

@harshvardhan178251 harshvardhan178251 commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes: #628

Zulip upload responses return a root-relative /user_uploads/... URL, and GET /api/v1/user_uploads/... returns a JSON temporary-access URL rather than the file, so call_endpoint() cannot be used to download uploads. This PR adds Client.fetch_upload(url_or_path, timeout=None):

  • Accepts a bare relative path (user_uploads/1/x.png), a server-root-relative path (/user_uploads/1/x.png), a /user_uploads/temporary/... token URL, or an absolute http(s) URL on this Zulip server. A leading /api/v1/ prefix is stripped (both for paths and absolute URLs), and the authenticated session follows the 302 to the file or the temporary URL.
  • Returns the file contents as bytes. Raises UnrecoverableNetworkError if the request can't be made and ZulipError if the server returns an HTTP error status.
  • Off-origin absolute URLs are rejected, so the bot's Basic Auth credentials are never sent to a third-party host.
  • HTTP statuses are left to the caller; no change to call_endpoint and no new API prefix behavior.

Design decision worth flagging: the /api/v1 prefix refactor suggested as "one option" in the issue is deliberately not included here — it would break existing bots that rely on call_endpoint() behaving as before. fetch_upload() is additive only; happy to revisit the refactor as a separate PR.

How did you test this PR?

  • New unit tests zulip/tests/test_fetch_upload.py (mocked HTTP layer): absolute-URL passthrough, root-relative path resolution against the server root, custom timeout, rejection of bare relative paths, non-200 → ZulipError.
  • Full zulip test suite passes locally (17 passed). ruff check and mypy clean. (The pre-existing zulip_bots/ failures were left untouched.)
Self-review checklist
  • Self-reviewed the changes for clarity and maintainability (variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

  • Explains differences from previous plans (e.g., issue description).
  • Highlights technical choices and bugs encountered.
  • Calls out remaining decisions and concerns.
  • Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

  • Each commit is a coherent idea.
  • Commit message(s) explain reasoning and motivation for changes.

@harshvardhan178251

Copy link
Copy Markdown
Collaborator Author

Hi — a quick status check on this issue from my side. My PR #948 adds Client.fetch_upload() (returns raw file bytes; absolute off-server URLs rejected so bot credentials are not sent elsewhere). All checks pass except two upstream-dead CI jobs (the bullseye Zulip 4.0 job, and Python 3.9 on the current ubuntu-latest runner), both of which are fixed in #949. I'm happy to address any feedback on the PR.

@Johnboard-cmyk

Copy link
Copy Markdown

Hi — I recently opened #950 for the same issue before noticing this PR and that the issue is claimed by you; I have closed mine in favor of yours. One thing worth a look, since it came up in the review of the earlier attempt #932: if fetch_upload accepts absolute URLs, the client's Basic Auth credentials would be sent to whatever host appears in the URL, so it is worth rejecting URLs whose host differs from the realm's host. My now-closed PR has one way of handling that, if it is helpful. Good luck with the PR!

Incorporate ideas from Johnboard-cmyk's closed PR zulip#950: strip a leading /api/v1/ prefix (people paste the wrong URL shape and hit the JSON temporary-URL instead of the file), accept bare relative paths under user_uploads/, and wrap connection failures in UnrecoverableNetworkError for consistency with do_api_query().
@harshvardhan178251

Copy link
Copy Markdown
Collaborator Author

Thanks @Johnboard-cmyk — much appreciated. The credential-leak concern you flagged was already implemented in this PR (off-origin absolute URLs are rejected without issuing the request; see zulip/tests/test_fetch_upload.py), and your comment made me double-check it holds. I've also folded in your other ideas in b14867e: a leading /api/v1/ prefix is now stripped (both for paths and full URLs, preserving the query string), bare relative paths under user_uploads/ are accepted, and connection failures are wrapped in UnrecoverableNetworkError for consistency with do_api_query(). Happy to keep iterating on feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add function for downloading an uploaded file

3 participants