Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 24 additions & 21 deletions apps/docs/content/api/canvas.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ description: Rendering pixels on-screen

| Prop | Type | Description |
| ---- | ---- | ----------- |
| `opaque` | `boolean` | Defaults to `true`. `false` alpha-composites the canvas over the views behind it (Android and web; iOS uses `alphaMode` alone) |
| `android` | `AndroidCanvasProps` | Android-only rendering options: `surfaceType` and `zOrderOnTop`. Ignored on iOS and web |
| `android` | `AndroidCanvasProps` | Android-only rendering options: the native view behind the canvas (`backingView`) and its options (`opaque`, `zOrderOnTop`, `zOrderMediaOverlay`), see [Android rendering options](#android-rendering-options). Ignored on iOS and web |
| `...ViewProps` | `ViewProps` | All standard React Native View props (`style`, `onLayout`, etc.) |

## Usage
Expand Down Expand Up @@ -109,7 +108,7 @@ export function TriangleScene() {
return (
<View style={{ flex: 1 }}>
<View style={{ flex: 1, backgroundColor: "#3498db" }} />
<Canvas ref={ref} style={StyleSheet.absoluteFill} opaque={false} />
<Canvas ref={ref} style={StyleSheet.absoluteFill} />
</View>
);
}
Expand All @@ -134,52 +133,56 @@ As on the Web, `clientWidth` / `clientHeight` are the layout size in points and

## Transparent compositing

To show the views behind the canvas, set `opaque={false}` on `<Canvas>`, configure the context with `alphaMode: "premultiplied"`, and clear with an alpha of `0`, as the [Usage](#usage) example does. On Android, `opaque` controls transparency and `alphaMode` is ignored; on iOS it is the other way around. Setting all three gives the same result on both platforms.
To show the views behind the canvas, configure the context with `alphaMode: "premultiplied"` and clear with an alpha of `0`, as the [Usage](#usage) example does. On iOS and on the web that is all it takes. On Android, `alphaMode` is ignored: the backing view decides, and the default `TextureView` composites with alpha unless `android={{ opaque: true }}` asks it not to (see below). Setting both gives the same result on every platform.

### Android rendering options

On Android, `Canvas` can be backed by one of three native views, with different trade-offs between speed and how well the canvas composites with the views around it. The defaults are right for most apps; the `android` prop selects one explicitly. It is ignored on iOS and web.
On Android, `Canvas` can be backed by one of three native views, with different trade-offs between speed and how well the canvas composites with the views around it. The `android` prop names it, along with its options; it is ignored on iOS and web.

| Option | Type | Effect |
| --- | --- | --- |
| `surfaceType` | `"SurfaceView" \| "TextureView" \| "HardwareBufferView"` | Backing view. Defaults to `SurfaceView` when `opaque` and to `TextureView` otherwise. `HardwareBufferView` is opt-in and needs Android 10 (API 29) |
| `backingView` | `"TextureView" \| "SurfaceView" \| "HardwareBufferView"` | The native view behind the canvas. Defaults to `TextureView` |
| `opaque` | `boolean` | Declares that the canvas covers every pixel of its bounds, so it is composited without blending. Defaults to `false` on a `TextureView` and to `true` on a `SurfaceView`; a `HardwareBufferView` always composites with alpha. Applied in place |
| `zOrderOnTop` | `boolean` | `SurfaceView` only: draws the canvas above every React Native view in the window. Defaults to `false` |
| `zOrderMediaOverlay` | `boolean` | `SurfaceView` only: draws the canvas above the other `SurfaceView`s below the window (video players, other canvases). Defaults to `false`; `zOrderOnTop` takes precedence |

| | `SurfaceView` | `TextureView` | `HardwareBufferView` |
| | `TextureView` | `SurfaceView` | `HardwareBufferView` |
| --- | --- | --- | --- |
| Minimum Android | 8 (API 26) | 8 (API 26) | 10 (API 29) |
| Behaves like a regular view | no | yes | yes |
| Latency | lowest | highest | in between |
| Status | default when `opaque` | default when `opaque={false}` | experimental, opt-in |
| Behaves like a regular view | yes | no | yes |
| Latency | highest | lowest | in between |
| Opaque by default | no | yes | never |
| Status | default | opt-in | experimental, opt-in |

**`SurfaceView`** is the fastest option, and the default for an opaque canvas. It is not regular view content, though: parent transforms, clipping, rounded corners and z-ordering with sibling views do not apply to it. Use it whenever the canvas is a plain opaque rectangle.
**`TextureView`** is the default: a regular view, so transforms, clipping, alpha and z-order all apply, and the canvas can be composited over other React Native views. The cost is an extra copy, typically a frame of latency, and occasional stalls during some view animations.

**`TextureView`** is a regular view, so transforms, clipping, alpha and z-order all apply, and the canvas can be composited over other React Native views. The cost is an extra copy, typically a frame of latency, and occasional stalls during some view animations.
**`SurfaceView`** is the fastest option. It is not regular view content, though: parent transforms, clipping, rounded corners and z-ordering with sibling views do not apply to it. Use it for a canvas that is a plain rectangle: a full-screen scene or a game. It is opaque unless asked otherwise: a translucent `SurfaceView` blends against the window background (usually black) instead of the React Native views beneath it, which are punched out. Pair it with `zOrderOnTop` to composite over React Native content, and note that `zOrderOnTop` draws above all React Native views in the window, including navigation screens, regardless of `zIndex`.

**`HardwareBufferView`** composites like a `TextureView`, with lower latency. It is experimental: if the device cannot allocate the buffers the canvas shows nothing instead of falling back, and some devices may show a stale or torn frame. Test it on your target devices, and drop the `surfaceType` to go back to `TextureView`. Below Android 10 it uses `TextureView`.
**`HardwareBufferView`** composites like a `TextureView`, with lower latency. It is experimental: if the device cannot allocate the buffers the canvas shows nothing instead of falling back, and some devices may show a stale or torn frame. Test it on your target devices, and drop the `backingView` to go back to `TextureView`. Below Android 10 it uses a `TextureView`, with a warning in development.

The backing view is chosen when the canvas mounts and kept while it is mounted. Changing `opaque` (and `zOrderOnTop` on Android 11 and later) updates it in place, but a change of `surfaceType` is ignored, with a warning in development. To switch backing views, remount the canvas with a `key`:
The backing view is created when the canvas mounts and kept while it is mounted. `opaque` applies in place, and so do the z-order options on Android 11 and later; a change of `backingView` is ignored, with a warning in development. To switch backing views, remount the canvas with a `key`. The remount leaves the canvas off screen for a frame or two, showing what is behind it, or black, until the new view draws its first frame: the old view leaves and the new one arrives in different window frames. Keeping a canvas on screen through such a swap is possible, but takes hold-and-retire logic specific to each Android release ([#563](https://github.com/wcandillon/react-native-webgpu/pull/563)) that the library does not carry.

```tsx
// The backing view is chosen at mount: a canvas that switches it remounts
<Canvas key={surfaceType} android={{ surfaceType }} />
<Canvas key={backingView} android={{ backingView }} />
```

Some combinations for other trade-offs:

```tsx
// Opaque TextureView: stays in stacking order, e.g. inside a ScrollView
<Canvas android={{ surfaceType: "TextureView" }} />
<Canvas android={{ opaque: true }} />

// Full-screen scene on a SurfaceView: no composition pass
<Canvas android={{ backingView: "SurfaceView" }} />

// Translucent SurfaceView above every React Native view
<Canvas opaque={false} android={{ surfaceType: "SurfaceView", zOrderOnTop: true }} />
<Canvas android={{ backingView: "SurfaceView", opaque: false, zOrderOnTop: true }} />

// Translucent HardwareBufferView, opt-in (uses TextureView before Android 10)
<Canvas opaque={false} android={{ surfaceType: "HardwareBufferView" }} />
// Translucent HardwareBufferView, opt-in (uses a TextureView before Android 10)
<Canvas android={{ backingView: "HardwareBufferView" }} />
```

A translucent `SurfaceView` without `zOrderOnTop` blends against the window background (usually black) instead of the React Native views beneath it. Pair it with `zOrderOnTop`, or use one of the two view-based options. Note that `zOrderOnTop` draws above all React Native views in the window, including navigation screens, regardless of `zIndex`.

## See also

- [GPUDeviceProvider](/api/gpu-device-provider) - share one device across components
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/content/api/draw-element-image-to-texture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ The snapshot is what the platform renders for that view hierarchy:
- iOS renders through `drawViewHierarchyInRect`, which includes Metal layers such as a WebGPU `Canvas`. With `RENDER_ATTACHMENT`, the pixels reach the texture without a copy through the CPU; otherwise they are drawn into a bitmap and uploaded.
- Android 10 (API 29) and above renders the views with the same renderer as the screen, so the result matches it pixel for pixel, including elevation shadows and `TextureView` content. Older versions draw the view into a software canvas, which misses content that is only drawn hardware-accelerated.
- Android 13 and above, with `RENDER_ATTACHMENT`: the pixels reach the texture without a copy through the CPU, and come out the same.
- On Android, `SurfaceView` content is composed outside the view hierarchy and comes out blank. A WebGPU `Canvas` uses a `SurfaceView` by default when opaque; give it `android={{ surfaceType: "TextureView" }}` if it must appear in a snapshot.
- On Android, `SurfaceView` content is composed outside the view hierarchy and comes out blank. A WebGPU `Canvas` with `android={{ backingView: "SurfaceView" }}` is not captured: keep the default `TextureView` for a canvas that must appear in a snapshot.

## Threading

Expand Down
11 changes: 4 additions & 7 deletions apps/docs/content/docs/getting-started/canvas.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The [`Canvas`](/api/canvas) component is where you can access a WebGPU context a
The API is symmetric with its Web counterpart, with a few differences.
For instance on React Native you must call `context.present()` after every `device.queue.submit()` to present the frame to the screen.

On Android, the canvas picks the right native view for you depending on whether it is opaque. The [Android rendering options](/api/canvas#android-rendering-options) explain the trade-offs and how to pick one explicitly.
On Android, the canvas is backed by a regular view by default. The [Android rendering options](/api/canvas#android-rendering-options) explain the trade-offs and how to pick a faster one.

| Web concept | React Native WebGPU |
| ----------- | --------------------- |
Expand Down Expand Up @@ -83,17 +83,14 @@ Because `clientWidth` / `clientHeight` track layout, recompute the buffer size w

## Transparency

To render the canvas over the React Native views beneath it, you control transparency from two places, and you normally set both:

- **`opaque={false}`** on `<Canvas>` makes the underlying native surface translucent so the views behind it show through.
- **`alphaMode: "premultiplied"`** on `context.configure`, together with a clear color whose alpha is `0`, makes WebGPU produce a transparent frame.
To render the canvas over the React Native views beneath it, configure the context with `alphaMode: "premultiplied"` and clear with a color whose alpha is `0`, so that WebGPU produces a transparent frame:

```tsx
<Canvas ref={ref} style={StyleSheet.absoluteFill} opaque={false} />
<Canvas ref={ref} style={StyleSheet.absoluteFill} />
```

<Callout type="info" title="Platform note">
On Android the `alphaMode` passed to `configure()` is ignored, so `opaque={false}` is what actually gives you a transparent canvas there. On iOS `alphaMode` alone controls it and `opaque` is ignored. Setting `opaque={false}` together with `alphaMode: "premultiplied"` and an alpha-`0` clear color gives you the same result on both platforms. See the [Android rendering options](/api/canvas#android-rendering-options) to pick the backing view explicitly.
On iOS `alphaMode` alone controls it. On Android the `alphaMode` passed to `configure()` is ignored: the backing view decides, and the default `TextureView` composites with alpha unless `android={{ opaque: true }}` asks it not to, while a `SurfaceView` is opaque unless asked otherwise. See the [Android rendering options](/api/canvas#android-rendering-options).
</Callout>

## See also
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/content/docs/integrations/typegpu.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ export function GradientTiles() {
ctx.present?.();
});

return <Canvas ref={ref} style={{ flex: 1 }} opaque={false} />;
return <Canvas ref={ref} style={{ flex: 1 }} />;
}
```

Expand Down
2 changes: 1 addition & 1 deletion apps/docs/src/demos/HelloTriangleDemo.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ export function HelloTriangleDemo({ transparent = true }: HelloTriangleDemoProps
<Canvas
ref={ref}
style={StyleSheet.absoluteFill}
opaque={!transparent}
android={{ opaque: !transparent }}
/>
</View>
);
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/src/demos/TransparencyDemo.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ export function TransparencyDemo() {
return (
<View style={styles.container}>
<Background />
<Canvas ref={ref} style={StyleSheet.absoluteFill} opaque={false} />
<Canvas ref={ref} style={StyleSheet.absoluteFill} />
</View>
);
}
Expand Down
3 changes: 1 addition & 2 deletions apps/example/src/Cube/Cube.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,7 @@ export function Cube() {
<Canvas
ref={ref}
style={style.webgpu}
opaque={false}
android={{ surfaceType: "HardwareBufferView" }}
android={{ backingView: "HardwareBufferView" }}
/>
</View>
);
Expand Down
22 changes: 13 additions & 9 deletions apps/example/src/Diagnostics/BackgroundDetach.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ import {
// second, and come back. Primarily an Android repro (iOS keeps the
// CAMetalLayer alive in the background).
//
// On a broken build the two Android view flavors fail differently:
// - non-opaque (TextureView): onSurfaceTextureDestroyed removes the surface
// On a broken build the two Android backing views fail differently:
// - TextureView (the default): onSurfaceTextureDestroyed removes the surface
// registry entry entirely. The resumed view registers a fresh entry the JS
// context has never seen, so the canvas stays black forever (and the
// context renders into the orphaned surface on a destroyed window).
// - opaque (SurfaceView): the surface detaches to offscreen and reattaches on
// - SurfaceView: the surface detaches to offscreen and reattaches on
// resume, but the reattach blit presents a frame the context never
// acquired (watch for a present-without-acquire validation error in the
// log) and permanently widens the configured usage with CopyDst.
Expand All @@ -31,7 +31,7 @@ import {
export const BackgroundDetach = () => {
const ref = useRef<CanvasRef>(null);
const { log, append } = useDiagnosticLog();
const [opaque, setOpaque] = useState(false);
const [surfaceView, setSurfaceView] = useState(false);

useEffect(() => {
const sub = AppState.addEventListener("change", (state) => {
Expand Down Expand Up @@ -76,17 +76,21 @@ export const BackgroundDetach = () => {
<View style={diagnosticStyles.controls}>
<Text style={diagnosticStyles.description}>
Background the app and return: the animated gradient must resume. On a
broken build the non-opaque canvas stays black forever (Android), and
the opaque one logs a validation error on resume.
broken build the TextureView canvas stays black forever (Android), and
the SurfaceView one logs a validation error on resume.
</Text>
<View style={styles.row}>
<Text style={diagnosticStyles.description}>
opaque (SurfaceView on Android)
SurfaceView (Android; the default is a TextureView)
</Text>
<Switch value={opaque} onValueChange={setOpaque} />
<Switch value={surfaceView} onValueChange={setSurfaceView} />
</View>
</View>
<Canvas ref={ref} style={diagnosticStyles.canvas} opaque={opaque} />
<Canvas
ref={ref}
style={diagnosticStyles.canvas}
android={{ backingView: surfaceView ? "SurfaceView" : "TextureView" }}
/>
<ScrollView style={diagnosticStyles.log}>
{log.map((line, i) => (
<Text key={i} style={diagnosticStyles.logLine}>
Expand Down
41 changes: 31 additions & 10 deletions apps/example/src/Diagnostics/DeviceDestroyBeforeDetach.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import React, { useEffect, useRef, useState } from "react";
import { Button, ScrollView, Switch, Text, View } from "react-native";
import type { CanvasRef } from "react-native-webgpu";
import type { AndroidCanvasProps, CanvasRef } from "react-native-webgpu";
import { Canvas } from "react-native-webgpu";
import * as THREE from "three";

Expand Down Expand Up @@ -58,12 +58,17 @@ type Mode = "raw" | "three";

interface SceneProps {
mode: Mode;
opaque: boolean;
surfaceView: boolean;
unconfigureFirst: boolean;
append: (line: string) => void;
}

const RawScene = ({ opaque, unconfigureFirst, append }: SceneProps) => {
// Either kind of Android backing view, to tear both down.
const backingView = (surfaceView: boolean): AndroidCanvasProps => ({
backingView: surfaceView ? "SurfaceView" : "TextureView",
});

const RawScene = ({ surfaceView, unconfigureFirst, append }: SceneProps) => {
const ref = useRef<CanvasRef>(null);

useEffect(() => {
Expand Down Expand Up @@ -122,10 +127,16 @@ const RawScene = ({ opaque, unconfigureFirst, append }: SceneProps) => {
};
}, [append, unconfigureFirst]);

return <Canvas ref={ref} style={diagnosticStyles.canvas} opaque={opaque} />;
return (
<Canvas
ref={ref}
style={diagnosticStyles.canvas}
android={backingView(surfaceView)}
/>
);
};

const ThreeScene = ({ opaque, append }: SceneProps) => {
const ThreeScene = ({ surfaceView, append }: SceneProps) => {
const ref = useRef<CanvasRef>(null);

useEffect(() => {
Expand Down Expand Up @@ -181,14 +192,20 @@ const ThreeScene = ({ opaque, append }: SceneProps) => {
};
}, [append]);

return <Canvas ref={ref} style={diagnosticStyles.canvas} opaque={opaque} />;
return (
<Canvas
ref={ref}
style={diagnosticStyles.canvas}
android={backingView(surfaceView)}
/>
);
};

export const DeviceDestroyBeforeDetach = () => {
const { log, append } = useDiagnosticLog();
const [mounted, setMounted] = useState(true);
const [mode, setMode] = useState<Mode>("raw");
const [opaque, setOpaque] = useState(true);
const [surfaceView, setSurfaceView] = useState(true);
const [unconfigureFirst, setUnconfigureFirst] = useState(false);
const Scene = mode === "raw" ? RawScene : ThreeScene;

Expand Down Expand Up @@ -216,9 +233,13 @@ export const DeviceDestroyBeforeDetach = () => {
/>
</View>
<View style={{ flexDirection: "row", alignItems: "center", gap: 8 }}>
<Switch value={opaque} onValueChange={setOpaque} disabled={mounted} />
<Switch
value={surfaceView}
onValueChange={setSurfaceView}
disabled={mounted}
/>
<Text style={diagnosticStyles.description}>
opaque (SurfaceView) / non-opaque (TextureView)
SurfaceView / TextureView
</Text>
</View>
{mode === "raw" ? (
Expand All @@ -237,7 +258,7 @@ export const DeviceDestroyBeforeDetach = () => {
{mounted ? (
<Scene
mode={mode}
opaque={opaque}
surfaceView={surfaceView}
unconfigureFirst={unconfigureFirst}
append={append}
/>
Expand Down
Loading
Loading