Textures

The following section describes the available texture data sources. In Mitsuba 3, textures are objects that can be attached to certain surface scattering model parameters to introduce spatial variation. In the documentation, these are listed as supporting the texture type. See the last sections about BSDFs for many examples.

Textures take an (optional) <transform> called to_uv which can be used to translate, scale, or rotate the lookup into the texture accordingly.

An example in XML looks as follows:

XMLPython

<scene version="3.0.0">
    <!-- Create a BSDF that supports textured parameters -->
    <bsdf type=".. BSDF type .." id="my_textured_material">
        <texture type=".. texture type .." name=".. parameter name ..">
            <!-- .. Texture parameters go here .. -->

            <transform name="to_uv">
                <!-- Scale texture by factor of 2 -->
                <scale x="2" y="2"/>
                <!-- Offset texture by [0.5, 1.0] -->
                <translate x="0.5" y="1.0"/>
            </transform>
        </texture>

        <!-- .. Non-spatially varying BSDF parameters ..-->
    </bsdf>
</scene>

Similar to BSDFs, named textures can alternatively be defined at the top level of the scene and later referenced. This is particularly useful if the same texture would be loaded many times otherwise.

XMLPython

<scene version="3.0.0">
    <!-- Create a named texture at the top level -->
    <texture type=".. texture type .." id="my_named_texture">
        <!-- .. Texture parameters go here .. -->
    </texture>

    <!-- Create a BSDF that supports textured parameters -->
    <bsdf type=".. BSDF type ..">
        <!-- Example of referencing a named texture -->
        <ref id="my_named_texture" name=".. parameter name .."/>

        <!-- .. Non-spatially varying BSDF parameters ..-->
    </bsdf>
</scene>

Bitmap texture (bitmap)

Parameter	Type	Description	Flags
filename	string	Filename of the bitmap to be loaded
bitmap	Bitmap object	When creating a Bitmap texture at runtime, e.g. from Python or C++, an existing Bitmap image instance can be passed directly rather than loading it from the filesystem with filename.
data	tensor	Tensor array containing the texture data. Similarly to the bitmap parameter, this field can only be used at runtime. The raw parameter must also be set to true.	P, ∂
filter_type	string	Specifies how pixel values are interpolated and filtered when queried over larger UV regions. The following options are currently available: bilinear (default): perform bilinear interpolation, but no filtering. nearest: disable filtering and interpolation. In this mode, the plugin performs nearest neighbor lookups of texture values. trilinear: build a MIP pyramid and blend the two levels that match the size of the lookup’s footprint. See the discussion of texture filtering below. anisotropic: like trilinear, but with additional taps along the major axis of the footprint, up to max_anisotropy. The two filtered modes are unsupported for color textures in spectral variants and fall back to bilinear there.
max_anisotropy	integer	Upper bound on the number of taps used by anisotropic filtering, in the range \([1, 16]\). (Default: 8)
wrap_mode	string	Controls the behavior of texture evaluations that fall outside of the \([0, 1]\) range. The following options are currently available: repeat (default): tile the texture infinitely. mirror: mirror the texture along its boundaries. clamp: clamp coordinates to the edge of the texture.
format	string	Specifies the underlying texture storage format. The following options are currently available: auto (default): Match the storage precision to the source: use 8 bits per channel for 8-bit images, half precision for 16-bit images, and otherwise the native floating point representation of the Mitsuba variant. For variants using a spectral color representation this option is the same as variant. Note that 8-bit storage is not differentiable; request float16 or variant to optimize such textures. variant: Use the corresponding native floating point representation of the Mitsuba variant float16: Store the texture in half precision uint8: Store the texture using 8 bits per channel. When the source image is sRGB-encoded and raw is false, the values are linearized on each lookup. This mode is the most memory efficient, but not that the texture is not differentiable. It is also incompatible with spectral variants of Mitsuba.
raw	boolean	Should the transformation to the stored color data (e.g. sRGB to linear, spectral upsampling) be disabled? You will want to enable this when working with bitmaps storing normal maps that use a linear encoding. (Default: false)
to_uv	transform	Specifies an optional 3x3 transformation matrix that will be applied to UV values. A 4x4 matrix can also be provided, in which case the extra row and column are ignored.	P
accel	boolean	Hardware acceleration features can be used in CUDA mode. These features can cause small differences as hardware interpolation methods typically have a loss of precision (not exactly 32-bit arithmetic). (Default: true)

This plugin provides a bitmap texture that performs interpolated lookups given a JPEG, PNG, OpenEXR, RGBE, TGA, or BMP input file.

When loading the plugin, the data is first converted into a usable color representation for the renderer:

• In rgb modes, sRGB textures are converted into a linear color space.

• In spectral modes, sRGB textures are spectrally upsampled to plausible smooth spectra [JH19] and stored an intermediate representation that enables efficient queries at render time.

• In monochrome modes, sRGB textures are converted to grayscale.

These conversions can alternatively be disabled with the raw flag, e.g. when textured data is already in linear space or does not represent colors at all.

Texture filtering. A distant or obliquely viewed texture covers many texels per pixel, which can introduce significant variance and aliasing, especially at low sampling rates. Mitsuba can optionally use ray cones [AMollerNA+19] to track the elliptical texture region visible within a pixel and perform filtered texture lookups that average over this footprint. Set filter_type to trilinear or anisotropic to enable this behavior. The trilinear filter reduces the footprint to a single level of detail, while the anisotropic filter takes several taps along its major axis, which is slower but produces better quality. Filtering currently only affects directly visible surfaces, and the cone_scale sensor parameter adjusts the footprint size.

Single-sample renderings of a checkerboard plane with the bilinear, trilinear, and anisotropic filter modes, each next to a converged reference. Unfiltered lookups alias near the horizon, trilinear filtering removes the aliasing but blurs the grazing view, and anisotropic filtering preserves most detail.

XMLPython

<texture type="bitmap">
    <string name="filename" value="texture.png"/>
    <string name="wrap_mode" value="mirror"/>
</texture>

Checkerboard texture (checkerboard)

Parameter	Type	Description	Flags
color0, color1	spectrum or texture	Color values for the two differently-colored patches (Default: 0.4 and 0.2)	P, ∂
to_uv	transform	Specifies an optional 3x3 UV transformation matrix. A 4x4 matrix can also be provided. In that case, the last row and columns will be ignored. (Default: none)	P

This plugin provides a simple procedural checkerboard texture with customizable colors.

Checkerboard applied to the material test object as well as the ground plane.¶

XMLPython

<texture type="checkerboard">
    <rgb name="color0" value="0.1, 0.1, 0.1"/>
    <rgb name="color1" value="0.5, 0.5, 0.5"/>
</texture>

Mesh attribute texture (mesh_attribute)

Parameter	Type	Description	Flags
name	string	Name of the attribute to evaluate. It should always start with "vertex_" or "face_".
scale	float	Scaling factor applied to the interpolated attribute value during evaluation. (Default: 1.0)	P

This plugin provides a simple mechanism to expose Mesh attributes (e.g. vertex color) as a texture.

Bunny with random vertex color (using barycentric interpolation).¶

Bunny with random face color.¶

The following XML snippet describes a mesh with diffuse material, whose reflectance is specified using the vertex_color attribute of that mesh:

XMLPython

<shape type="ply">
    <string name="filename" value="my_mesh_with_vertex_color_attr.ply"/>

    <bsdf type="diffuse">
        <texture type="mesh_attribute" name="reflectance">
            <string name="name" value="vertex_color"/>
        </texture>
    </bsdf>
</shape>

Note

For spectral variants of the renderer (e.g. scalar_spectral), when a mesh attribute name contains the string "color", the tri-stimulus RGB values will be converted to rgb2spec model coefficients automatically.

Volumetric texture (volume)

Parameter	Type	Description	Flags
volume	float, spectrum or volume	Volumetric texture (Default: 0.75).	P, ∂

This plugin allows using a 3D texture (i.e. a volume plugin) to texture a 2D surface. This is intended to be used to texture surfaces without a meaningful UV parameterization (e.g., an implicit surface) or to apply procedural 3D textures. At a given point on a surface, the texture value will be determined by looking up the corresponding value in the referenced volume. This is done in world space and potentially requires using the volume’s to_world transformation to align the volume with the object using the texture.

XMLPython

<texture type="volume">
    <volume name="volume" type="gridvolume">
        <string name="filename" value="my_volume.vol"/>
    </volume>
</texture>

Math expression texture (math)

Parameter	Type	Description	Flags
expr	string	Expression to evaluate using the simple language described below.
(Nested plugin)	texture	Input textures. The expression refers to them as in[0], in[1], etc., numbered in the order of declaration. In the scene parameter API, they are exposed as differentiable parameters named in0, in1, …	P, ∂

This texture evaluates a mathematical expression involving an arbitrary number of input textures. This can be helpful to transform textures in ways that are not covered by any existing plugin.

The expression consists of an optional sequence of assignments to temporaries tmp[0], tmp[1], …, separated by semicolons, followed by a final expression that produces the result of the texture. Each temporary must be assigned exactly once before use. For example:

tmp[0] = clip(in[0], 0, 1); lerp(in[1], in[2], tmp[0])

The language provides

• The arithmetic operators +, -, *, /, unary -, and parentheses.

• Comparisons <, <=, >, >=, ==, != and the ternary operator cond ? a : b to consume them.

• The logical operators &&, ||, and !, which treat any nonzero operand as true.

• Unary functions: abs, sign, sqrt, cbrt, rcp, rsqrt, erf, sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, asinh, acosh, atanh, exp, log, exp2, log2, round, trunc, floor, ceil.

• Binary and ternary functions: min, max, pow, atan2, fmod, lerp(a, b, t), clip(x, lo, hi), fma(a, b, c).

• Color handling: rgb(r, g, b) assembles a trichromatic value, and the suffixes .r, .g, and .b extract a single channel of an expression and broadcast it to all channels. The functions mean and luminance reduce a value to its channel average or ITU-R BT.709 luminance, which they likewise broadcast to all channels.

• Floating point literals and the constants pi and e.

Operator precedence follows the C language. The language only supports floating point values and represents Boolean values as 0.0 and 1.0. Monochromatic, trichromatic, and spectral texture queries apply the expression separately to each input channel.

The color handling operations are the exception to the last rule. Channel k of rgb(r, g, b) evaluates to channel k of its k-th argument, and the component suffixes copy one channel to all others. In monochromatic queries, component extraction and luminance return their argument unchanged, and rgb reduces to the luminance of its three arguments. Spectral queries in spectral variants process wavelength samples that lack a channel interpretation. There, mean averages the wavelength samples, while rgb, component extraction, and luminance raise an error.

The plugin compiles the expression into a compact stack machine bytecode representation that is interpreted during texture evaluation. In JIT-compiled variants, this interpretation runs once at kernel trace time, hence the evaluation cost matches that of directly written arithmetic.

XMLPython

<texture type="math">
    <string name="expr" value="in[0] * (1 - in[1])"/>
    <texture type="bitmap">
        <string name="filename" value="texture.png"/>
    </texture>
    <texture type="checkerboard"/>
</texture>