page.title=Graphics parent.title=Renderscript parent.link=index.html @jd:body
Renderscript provides a number of graphics APIs for rendering, both at the Android framework level as well as at the Renderscript runtime level. For instance, the Android framework APIs let you create meshes and define shaders to customize the graphical rendering pipeline. The native Renderscript graphics APIs let you draw the actual meshes to render your scene. You need to be familiar with both APIs to appropriately render graphics on an Android-powered device.
Renderscript applications require various layers of code, so it is useful to create the following files to help keep your application organized:
.rs
file.java
class.rs
file. This class contains a Renderscript object (instance of
ScriptC_renderscript_file
), which allows your Android framework code to
call the Renderscript code. In general, this class does much of the setup for Renderscript
such as shader and mesh building and memory allocation and binding. The SDK samples follow the
convention of naming this file ActivityRS.java,
where Activity is the name of your main activity class..java
class.java
classFigure 1 describes how these classes interact with one another in a graphics Renderscript:
Figure 1. Graphics Renderscript overview
The following sections describe how to create an application that uses a graphics Renderscript by using the Renderscript Fountain sample that is provided in the SDK as a guide (some code has been modified from its original form for simplicity).
Your Renderscript code resides in .rs
and .rsh
(headers) files in the
<project_root>/src/
directory. This code contains the logic to render your
graphics and declares all other necessary items such as variables, structs,
and pointers. Every graphics .rs
file generally contains the following items:
#pragma rs java_package_name(package.name)
) that declares
the package name of the .java
reflection of this Renderscript.#pragma version(1)
) that declares the version of Renderscript that
you are using (1 is the only value for now).#include "rs_graphics.rsh"
declaration.root()
function. This is the main worker function for your Renderscript and
calls Renderscript graphics functions to render scenes. This function is called every time a
frame refresh occurs, which is specified as its return value. A 0
(zero) specified for
the return value says to only render the frame when a property of the scene that you are
rendering changes. A non-zero positive integer specifies the refresh rate of the frame in
milliseconds.
Note: The Renderscript runtime makes its best effort to refresh the frame at the specified rate. For example, if you are creating a live wallpaper and set the return value to 20, the Renderscript runtime renders the wallpaper at 50fps if it has just enough or more resources to do so. It renders as fast as it can if not enough resources are available.
For more information on using the Renderscript graphics functions, see the Drawing section.
init()
function. This allows you to do initialization of your
Renderscript before the root()
function runs, such as assigning values to variables. This
function runs once and is called automatically when the Renderscript starts, before anything
else in your Renderscript. Creating this function is optional..rsh
files if desired)The following code shows how the fountain.rs
file is implemented:
#pragma version(1) // Tell which java package name the reflected files should belong to #pragma rs java_package_name(com.example.android.rs.fountain) //declare shader binding #pragma stateFragment(parent) // header with graphics APIs, must include explicitly #include "rs_graphics.rsh" static int newPart = 0; // the mesh to render rs_mesh partMesh; // the point representing where a particle is rendered typedef struct __attribute__((packed, aligned(4))) Point { float2 delta; float2 position; uchar4 color; } Point_t; Point_t *point; // main worker function that renders particles onto the screen int root() { float dt = min(rsGetDt(), 0.1f); rsgClearColor(0.f, 0.f, 0.f, 1.f); const float height = rsgGetHeight(); const int size = rsAllocationGetDimX(rsGetAllocation(point)); float dy2 = dt * (10.f); Point_t * p = point; for (int ct=0; ct < size; ct++) { p->delta.y += dy2; p->position += p->delta; if ((p->position.y > height) && (p->delta.y > 0)) { p->delta.y *= -0.3f; } p++; } rsgDrawMesh(partMesh); return 1; } // adds particles to the screen to render static float4 partColor[10]; void addParticles(int rate, float x, float y, int index, bool newColor) { if (newColor) { partColor[index].x = rsRand(0.5f, 1.0f); partColor[index].y = rsRand(1.0f); partColor[index].z = rsRand(1.0f); } float rMax = ((float)rate) * 0.02f; int size = rsAllocationGetDimX(rsGetAllocation(point)); uchar4 c = rsPackColorTo8888(partColor[index]); Point_t * np = &point[newPart]; float2 p = {x, y}; while (rate--) { float angle = rsRand(3.14f * 2.f); float len = rsRand(rMax); np->delta.x = len * sin(angle); np->delta.y = len * cos(angle); np->position = p; np->color = c; newPart++; np++; if (newPart >= size) { newPart = 0; np = &point[newPart]; } } }
When you create a Renderscript (.rs
) file, it is helpful to create a
corresponding Android framework class that is an entry point into the .rs
file.
The most important thing this class does is receive a {@link android.renderscript.RenderScriptGL} rendering context
object from the view class and binds the actual Renderscript
code to the rendering context. This notifies your view class of the code that it needs
to render graphics.
In addition, this class should contain all of the things needed to set up Renderscript. Some important things that you need to do in this class are:
ScriptC_rs_filename
. The Renderscript object is attached to the Renderscript bytecode, which is platform-independent and
gets compiled on the device when the Renderscript application runs. The bytecode is referenced
as a raw resource and is passed into the constructor for the Renderscript object.
For example, this is how the Fountain
sample creates the Renderscript object:
RenderScriptGL rs; //obtained from the view class Resources res; //obtained from the view class ... ScriptC_fountain mScript = new ScriptC_fountain(mRS, mRes, R.raw.fountain);
The following code shows how the FountainRS class is implemented:
package com.example.android.rs.fountain; import android.content.res.Resources; import android.renderscript.*; import android.util.Log; public class FountainRS { public static final int PART_COUNT = 50000; public FountainRS() { } /** * This provides us with the Renderscript context and resources * that allow us to create the Renderscript object */ private Resources mRes; private RenderScriptGL mRS; // Renderscript object private ScriptC_fountain mScript; // Called by the view class to initialize the Renderscript context and renderer public void init(RenderScriptGL rs, Resources res) { mRS = rs; mRes = res; /** * Create a shader and bind to the Renderscript context */ ProgramFragmentFixedFunction.Builder pfb = new ProgramFragmentFixedFunction.Builder(rs); pfb.setVaryingColor(true); rs.bindProgramFragment(pfb.create()); /** * Allocate memory for the particles to render and create the mesh to draw */ ScriptField_Point points = new ScriptField_Point(mRS, PART_COUNT); Mesh.AllocationBuilder smb = new Mesh.AllocationBuilder(mRS); smb.addVertexAllocation(points.getAllocation()); smb.addIndexSetType(Mesh.Primitive.POINT); Mesh sm = smb.create(); /** * Create and bind the Renderscript object to the Renderscript context */ mScript = new ScriptC_fountain(mRS, mRes, R.raw.fountain); mScript.set_partMesh(sm); mScript.bind_point(points); mRS.bindRootScript(mScript); } boolean holdingColor[] = new boolean[10]; /** * Calls Renderscript functions (invoke_addParticles) * via the Renderscript object to add particles to render * based on where a user touches the screen. */ public void newTouchPosition(float x, float y, float pressure, int id) { if (id >= holdingColor.length) { return; } int rate = (int)(pressure * pressure * 500.f); if (rate > 500) { rate = 500; } if (rate > 0) { mScript.invoke_addParticles(rate, x, y, id, !holdingColor[id]); holdingColor[id] = true; } else { holdingColor[id] = false; } } }
To display graphics, you need a view to render on. Create a class that extends {@link
android.renderscript.RSSurfaceView} or {@link android.renderscript.RSTextureView}. This class
allows you to create a {@link android.renderscript.RenderScriptGL} context object by calling and
pass it to the Rendscript entry point class to bind the two. Once bound, the content is aware
of the code that it needs to use to render graphics with. If your Renderscript code
depends on any type of information that the view is aware of, such as touches from the user,
you can also use this class to relay that information to the Renderscript entry point class.
The following code shows how the FountainView
class is implemented:
package com.example.android.rs.fountain; import android.renderscript.RSTextureView; import android.renderscript.RenderScriptGL; import android.content.Context; import android.view.MotionEvent; public class FountainView extends RSTextureView { public FountainView(Context context) { super(context); } // Renderscript context private RenderScriptGL mRS; // Renderscript entry point object that calls Renderscript code private FountainRS mRender; /** * Create Renderscript context and initialize Renderscript entry point */ @Override protected void onAttachedToWindow() { super.onAttachedToWindow(); android.util.Log.e("rs", "onAttachedToWindow"); if (mRS == null) { RenderScriptGL.SurfaceConfig sc = new RenderScriptGL.SurfaceConfig(); mRS = createRenderScriptGL(sc); mRender = new FountainRS(); mRender.init(mRS, getResources()); } } @Override protected void onDetachedFromWindow() { super.onDetachedFromWindow(); android.util.Log.e("rs", "onDetachedFromWindow"); if (mRS != null) { mRS = null; destroyRenderScriptGL(); } } /** * Use callbacks to relay data to Renderscript entry point class */ @Override public boolean onTouchEvent(MotionEvent ev) { int act = ev.getActionMasked(); if (act == ev.ACTION_UP) { mRender.newTouchPosition(0, 0, 0, ev.getPointerId(0)); return false; } else if (act == MotionEvent.ACTION_POINTER_UP) { // only one pointer going up, we can get the index like this int pointerIndex = ev.getActionIndex(); int pointerId = ev.getPointerId(pointerIndex); mRender.newTouchPosition(0, 0, 0, pointerId); } int count = ev.getHistorySize(); int pcount = ev.getPointerCount(); for (int p=0; p < pcount; p++) { int id = ev.getPointerId(p); mRender.newTouchPosition(ev.getX(p), ev.getY(p), ev.getPressure(p), id); for (int i=0; i < count; i++) { mRender.newTouchPosition(ev.getHistoricalX(p, i), ev.getHistoricalY(p, i), ev.getHistoricalPressure(p, i), id); } } return true; } }
Applications that use Renderscript still behave like normal Android applications, so you need an activity class that handles activity lifecycle callback events appropriately. The activity class also sets your {@link android.renderscript.RSSurfaceView} view class to be the main content view of the activity or uses your {@link android.renderscript.RSTextureView} in a {@link android.view.ViewGroup} alongside other views.
The following code shows how the Fountain sample declares its activity class:
package com.example.android.rs.fountain; import android.app.Activity; import android.os.Bundle; import android.util.Log; public class Fountain extends Activity { private static final String LOG_TAG = "libRS_jni"; private static final boolean DEBUG = false; private static final boolean LOG_ENABLED = false; private FountainView mView; @Override public void onCreate(Bundle icicle) { super.onCreate(icicle); // Create our Preview view and set it as // the content of our activity mView = new FountainView(this); setContentView(mView); } @Override protected void onResume() { Log.e("rs", "onResume"); // Ideally a game should implement onResume() and onPause() // to take appropriate action when the activity looses focus super.onResume(); mView.resume(); } @Override protected void onPause() { Log.e("rs", "onPause"); // Ideally a game should implement onResume() and onPause() // to take appropriate action when the activity looses focus super.onPause(); mView.pause(); } static void log(String message) { if (LOG_ENABLED) { Log.v(LOG_TAG, message); } } }
Now that you have an idea of what is involved in a Renderscript graphics application, you can start building your own. It might be easiest to begin with one of the Renderscript samples as a starting point if this is your first time using Renderscript.
The following sections describe how to use the graphics functions to draw with Renderscript.
The native Renderscript APIs provide a few convenient functions to easily draw a polygon or text to
the screen. You call these in your root()
function to have them render to the {@link
android.renderscript.RSSurfaceView} or {@link android.renderscript.RSTextureView}. These functions are
available for simple drawing and should not be used for complex graphics rendering:
rsgDrawRect()
: Sets up a mesh and draws a rectangle to the screen. It uses the
top left vertex and bottom right vertex of the rectangle to draw.rsgDrawQuad()
: Sets up a mesh and draws a quadrilateral to the screen.rsgDrawQuadTexCoords()
: Sets up a mesh and draws a quadrilateral to the screen
using the provided coordinates of a texture.rsgDrawText()
: Draws specified text to the screen. Use rsgFontColor()
to set the color of the text.When you want to render complex scenes to the screen, instantiate a {@link
android.renderscript.Mesh} and draw it with rsgDrawMesh()
. A {@link
android.renderscript.Mesh} is a collection of allocations that represent vertex data (positions,
normals, texture coordinates) and index data that provides information on how to draw triangles
and lines with the provided vertex data. You can build a Mesh in three different ways:
To create a mesh using the {@link android.renderscript.Mesh.TriangleMeshBuilder}, you need to supply it with a set of vertices and the indices for the vertices that comprise the triangle. For example, the following code specifies three vertices, which are added to an internal array, indexed in the order they were added. The call to {@link android.renderscript.Mesh.TriangleMeshBuilder#addTriangle addTriangle()} draws the triangle with vertex 0, 1, and 2 (the vertices are drawn counter-clockwise).
int float2VtxSize = 2; Mesh.TriangleMeshBuilder triangles = new Mesh.TriangleMeshBuilder(renderscriptGL, float2VtxSize, Mesh.TriangleMeshBuilder.COLOR); triangles.addVertex(300.f, 300.f); triangles.addVertex(150.f, 450.f); triangles.addVertex(450.f, 450.f); triangles.addTriangle(0 , 1, 2); Mesh smP = triangle.create(true); script.set_mesh(smP);
To draw a mesh using the {@link android.renderscript.Mesh.AllocationBuilder}, you need to supply it with one or more allocations that contain the vertex data:
Allocation vertices; ... Mesh.AllocationBuilder triangle = new Mesh.AllocationBuilder(mRS); smb.addVertexAllocation(vertices.getAllocation()); smb.addIndexSetType(Mesh.Primitive.TRIANGLE); Mesh smP = smb.create(); script.set_mesh(smP);
In your Renderscript code, draw the built mesh to the screen:
rs_mesh mesh; ... int root(){ ... rsgDrawMesh(mesh); ... return 0; //specify a non zero, positive integer to specify the frame refresh. //0 refreshes the frame only when the mesh changes. }
You can attach four program objects to the {@link android.renderscript.RenderScriptGL} context to customize the rendering pipeline. For example, you can create vertex and fragment shaders in GLSL or build a raster program object that controls culling. The four programs mirror a traditional graphical rendering pipeline:
Android Object Type | Renderscript Native Type | Description |
---|---|---|
{@link android.renderscript.ProgramVertex} | rs_program_vertex |
The Renderscript vertex program, also known as a vertex shader, describes the stage in the graphics pipeline responsible for manipulating geometric data in a user-defined way. The object is constructed by providing Renderscript with the following data:
Once the program is created, bind it to the {@link android.renderscript.RenderScriptGL} graphics context by calling {@link android.renderscript.RenderScriptGL#bindProgramVertex bindProgramVertex()}. It is then used for all subsequent draw calls until you bind a new program. If the program has constant inputs, the user needs to bind an allocation containing those inputs. The allocation's type must match the one provided during creation. The Renderscript runtime then does all the necessary plumbing to send those constants to the graphics hardware. Varying inputs to the shader, such as position, normal, and texture coordinates are matched by name between the input {@link android.renderscript.Element} and the mesh object that is being drawn. The signatures don't have to be exact or in any strict order. As long as the input name in the shader matches a channel name and size available on the mesh, the Renderscript runtime handles connecting the two. Unlike OpenGL there is no need to link the vertex and fragment programs. To bind shader constants to the program, declare a The {@link android.renderscript.ProgramVertexFixedFunction.Builder} class also lets you build a simple vertex shader without writing GLSL code. |
{@link android.renderscript.ProgramFragment} | rs_program_fragment |
The Renderscript fragment program, also known as a fragment shader, is responsible for
manipulating pixel data in a user-defined way. It's constructed from a GLSL shader string
containing the program body, texture inputs, and a {@link android.renderscript.Type}
object that describes the constants
used by the program. Like the vertex programs, when an {@link android.renderscript.Allocation}
with constant input
values is bound to the shader, its values are sent to the graphics program automatically.
Note that the values inside the {@link android.renderscript.Allocation} are not explicitly tracked.
If they change between two draw calls using the same program object, notify the runtime of that change by
calling To bind shader constructs to the program, declare a The {@link android.renderscript.ProgramFragmentFixedFunction.Builder} class also lets you build a simple fragment shader without writing GLSL code. |
{@link android.renderscript.ProgramStore} | rs_program_store | The Renderscript store program contains a set of parameters that control how the graphics hardware writes to the framebuffer. It could be used to enable and disable depth writes and testing, setup various blending modes for effects like transparency and define write masks for color components. |
{@link android.renderscript.ProgramRaster} | rs_program_raster | The Renderscript raster program is primarily used to specify whether point sprites are enabled and to control the culling mode. By default back faces are culled. |
The following example defines a vertex shader in GLSL and binds it to a Renderscript context object:
private RenderScriptGL glRenderer; //rendering context private ScriptField_Point mPoints; //vertices private ScriptField_VpConsts mVpConsts; //shader constants ... ProgramVertex.Builder sb = new ProgramVertex.Builder(glRenderer); String t = "varying vec4 varColor;\n" + "void main() {\n" + " vec4 pos = vec4(0.0, 0.0, 0.0, 1.0);\n" + " pos.xy = ATTRIB_position;\n" + " gl_Position = UNI_MVP * pos;\n" + " varColor = vec4(1.0, 1.0, 1.0, 1.0);\n" + " gl_PointSize = ATTRIB_size;\n" + "}\n"; sb.setShader(t); sb.addConstant(mVpConsts.getType()); sb.addInput(mPoints.getElement()); ProgramVertex pvs = sb.create(); pvs.bindConstants(mVpConsts.getAllocation(), 0); glRenderer.bindProgramVertex(pvs);
The RsRenderStatesRS sample has many examples on how to create a shader without writing GLSL.
You can also declare four pragmas that control default program bindings to the {@link android.renderscript.RenderScriptGL} context when the script is executing:
stateVertex
stateFragment
stateRaster
stateStore
The possible values for each pragma are parent
or default
. Using
default
binds the shaders to the graphical context with the system defaults.
Using parent
binds the shaders in the same manner as it is bound in the calling
script. If this is the root script, the parent state is taken from the bind points that are set
by the {@link android.renderscript.RenderScriptGL} bind methods.
For example, you can define this at the top of your graphics Renderscript code to have the vertex and store programs inherent the bind properties from their parent scripts:
#pragma stateVertex(parent) #pragma stateStore(parent)
A {@link android.renderscript.Sampler} object defines how data is extracted from textures. Samplers are bound to a {@link android.renderscript.ProgramFragment} alongside the texture whose sampling they control. These objects are used to specify such things as edge clamping behavior, whether mip-maps are used, and the amount of anisotropy required. There might be situations where hardware does not support the desired behavior of the sampler. In these cases, the Renderscript runtime attempts to provide the closest possible approximation. For example, the user requested 16x anisotropy, but only 8x was set because it's the best available on the hardware.
The RsRenderStatesRS sample has many examples on how to create a sampler and bind it to a Fragment program.
Framebuffer objects allow you to render offscreen instead of in the default onscreen framebuffer. This approach might be useful for situations where you need to post-process a texture before rendering it to the screen, or when you want to composite two scenes in one such as rendering a rear-view mirror of a car. There are two buffers associated with a framebuffer object: a color buffer and a depth buffer. The color buffer (required) contains the actual pixel data of the scene that you are rendering, and the depth buffer (optional) contains the values necessary to figure out what vertices are drawn depending on their z-values.
In general, you need to do the following to render to a framebuffer object:
rsgBindColorTarget()
and passing it the color buffer
allocation. If applicable, call rsgBindDepthTarget()
passing in the depth buffer
allocation as well.rsgDraw
functions. The scene will be
rendered into the color buffer instead of the default onscreen framebuffer.rsgClearAllRenderTargets()
.The following example shows you how to render to a framebuffer object by modifying the Fountain Renderscript sample. The end result is the FountainFBO sample. The modifications render the exact same scene into a framebuffer object as it does the default framebuffer. The framebuffer object is then rendered into the default framebuffer in a small area at the top left corner of the screen.
fountain.rs
and add the following global variables. This creates setter
methods when this file is reflected into a .java
file, allowing you to allocate
memory in your Android framework code and binding it to the Renderscript runtime.
//allocation for color buffer rs_allocation gColorBuffer; //fragment shader for rendering without a texture (used for rendering to framebuffer object) rs_program_fragment gProgramFragment; //fragment shader for rendering with a texture (used for rendering to default framebuffer) rs_program_fragment gTextureProgramFragment;
fountain.rs
to look like the following code. The
modifications are commented:
int root() { float dt = min(rsGetDt(), 0.1f); rsgClearColor(0.f, 0.f, 0.f, 1.f); const float height = rsgGetHeight(); const int size = rsAllocationGetDimX(rsGetAllocation(point)); float dy2 = dt * (10.f); Point_t * p = point; for (int ct=0; ct < size; ct++) { p->delta.y += dy2; p->position += p->delta; if ((p->position.y > height) && (p->delta.y > 0)) { p->delta.y *= -0.3f; } p++; } //Tell Renderscript runtime to render to the frame buffer object rsgBindColorTarget(gColorBuffer, 0); //Begin rendering on a white background rsgClearColor(1.f, 1.f, 1.f, 1.f); rsgDrawMesh(partMesh); //When done, tell Renderscript runtime to stop rendering to framebuffer object rsgClearAllRenderTargets(); //Bind a new fragment shader that declares the framebuffer object to be used as a texture rsgBindProgramFragment(gTextureProgramFragment); //Bind the framebuffer object to the fragment shader at slot 0 as a texture rsgBindTexture(gTextureProgramFragment, 0, gColorBuffer); //Draw a quad using the framebuffer object as the texture float startX = 10, startY = 10; float s = 256; rsgDrawQuadTexCoords(startX, startY, 0, 0, 1, startX, startY + s, 0, 0, 0, startX + s, startY + s, 0, 1, 0, startX + s, startY, 0, 1, 1); //Rebind the original fragment shader to render as normal rsgBindProgramFragment(gProgramFragment); //Render the main scene rsgDrawMesh(partMesh); return 1; }
FountainRS.java
file, modify the init()
method to look
like the following code. The modifications are commented:
/* Add necessary members */ private ScriptC_fountainfbo mScript; private Allocation mColorBuffer; private ProgramFragment mProgramFragment; private ProgramFragment mTextureProgramFragment; public void init(RenderScriptGL rs, Resources res) { mRS = rs; mRes = res; ScriptField_Point points = new ScriptField_Point(mRS, PART_COUNT); Mesh.AllocationBuilder smb = new Mesh.AllocationBuilder(mRS); smb.addVertexAllocation(points.getAllocation()); smb.addIndexSetType(Mesh.Primitive.POINT); Mesh sm = smb.create(); mScript = new ScriptC_fountainfbo(mRS, mRes, R.raw.fountainfbo); mScript.set_partMesh(sm); mScript.bind_point(points); ProgramFragmentFixedFunction.Builder pfb = new ProgramFragmentFixedFunction.Builder(rs); pfb.setVaryingColor(true); mProgramFragment = pfb.create(); mScript.set_gProgramFragment(mProgramFragment); /* Second fragment shader to use a texture (framebuffer object) to draw with */ pfb.setTexture(ProgramFragmentFixedFunction.Builder.EnvMode.REPLACE, ProgramFragmentFixedFunction.Builder.Format.RGBA, 0); /* Set the fragment shader in the Renderscript runtime */ mTextureProgramFragment = pfb.create(); mScript.set_gTextureProgramFragment(mTextureProgramFragment); /* Create the allocation for the color buffer */ Type.Builder colorBuilder = new Type.Builder(mRS, Element.RGBA_8888(mRS)); colorBuilder.setX(256).setY(256); mColorBuffer = Allocation.createTyped(mRS, colorBuilder.create(), Allocation.USAGE_GRAPHICS_TEXTURE | Allocation.USAGE_GRAPHICS_RENDER_TARGET); /* Set the allocation in the Renderscript runtime */ mScript.set_gColorBuffer(mColorBuffer); mRS.bindRootScript(mScript); }
Note: This sample doesn't use a depth buffer, but the following code shows you how to declare an example depth buffer if you need to use one for your application. The depth buffer must have the same dimensions as the color buffer:
Allocation mDepthBuffer; ... Type.Builder b = new Type.Builder(mRS, Element.createPixel(mRS, DataType.UNSIGNED_16, DataKind.PIXEL_DEPTH)); b.setX(256).setY(256); mDepthBuffer = Allocation.createTyped(mRS, b.create(), Allocation.USAGE_GRAPHICS_RENDER_TARGET);