Creating SImage¶
Editor Style¶
、
To use an existing image in EditorStyle, you can specify it with "Brush". For example:
"SImage": {
"Image": {
"Style": "FEditorStyle",
"Brush": "AutomationTools.TestAutomation"
}
}
NOTE
EditorStyles supported in TAPython include: "FEditorStyle", "FCoreStyle", and "FAppStyle events".
ImagePath¶
The "ImagePath" field can be used to specify the image path. This path is relative to the current UI JSON file of the tool.
For example:
"SImage": {
"Aka": "AUniqueName",
"ColorAndOpacity": [1, 1, 1, 1],
"ImagePath": "../ChameleonGallery/Images/cartoon_game_map.png",
"ToolTipText": "'ImagePath' is a relative path with the local ui json file.",
"OnMouseButtonDown": "print('clicked')"
}
For images used in the current tool, it is convenient to place them in the tool's directory or its subdirectories.
ImagePathInPlugin¶
To use an image located in the plugin directory, you can specify it with "ImagePathInPlugin". The path is also relative, relative to the TAPython directory. For example:
"ImagePathInPlugin": "Resources/Icon128.png"，the actual path is <Your_Project>\Plugins\TAPython\Resources/Icon128.png.
"SImage": {
"Aka": "AUniqueName",
"ImagePathInPlugin": "Resources/Icon128.png",
"DesiredSizeOverride": [64, 64]
}
CAUTION
When specifying image content in SImage through "ImagePath" and "ImagePathInPlugin", you also need to specify the "Aka" widget name.
SetImageFromPath¶
unreal.ChameleonData.set_image_from_path(aka_name, image_file_path, brush_width, brush_height)，can be used to set the image content of SImage from the image path.
aka_name: The name of the SImage widget.image_file_path: The absolute path of the image. (If a relative path is provided, it is relative to the Resource directory in the Plugin directory)brush_width: The width of the generated Brushbrush_height: The height of the generated Brush
CAUTION
Using relative paths in image_file_path increases the memory overhead for developers. It is recommended to use absolute paths when using set_image_from_path.
Fill SImage¶
In addition to the content introduced below, more details on filling and modifying SImage can be found here: Image and Render Target
set_image_pixel¶
chameleon_instance.set_image_pixels(aka_name, pixel_colors, width, height), can be used to fill the image content of SImage with pixels.
chameleon_instance：An instance of ChameleonData.
For example, the following code sets the image content of SImage to a 64x64 random black and white noise map.
def set_random_image_data(self):
width = 64
height = 64
colors = [unreal.LinearColor(1, 1, 1, 1) if random.randint(0, 1) else unreal.LinearColor(0, 0, 0, 1) for _ in range(width * height)]
self.data.set_image_pixels(self.ui_image, colors, width, height)
set_image_data¶
Filling SImage with pixels is very intuitive, but in practice, we often use "raw_data" to fill the content of SImage. Its execution speed is relatively faster.
chameleon_instance.set_image_data(aka_name, raw_data, width, height, channel_num=4, bgr=True)
raw_data: The flatten raw data of image, len(RawDataIn) == Height * Width * ChannelNum
For example, in TAPython_Taichi_StableFluid_UE5, numpy is used as the data structure for images. We can fill the image into SImage with the following code.
import numpy as np
...
self.image_im = np.ones((self.height, self.width, 4), dtype=np.uint8) * 255
...
self.data.set_image_data(self.ui_image, self.image_im.data.tobytes(), width=self.width, height=self.height)
Another example: We got a base64 encoded string of an image from the network. We can fill it into SImage with the following code.
from PIL import Image
def on_timer(self, count):
if count != 0 and count != -1:
progress_json = self.api.get_progress()
progress = progress_json.get("progress", None)
current_image = progress_json.get("current_image", None)
if progress:
self._set_progress(progress)
if current_image:
img = Image.open(io.BytesIO(base64.b64decode(current_image)))
self.data.set_image_data(self.ui_image, img.tobytes(), img.width, img.height, channel_num=3, bgr=False)
TIP
Third-party libraries like numpy and PIL can greatly speed up image calculations. GPU calculations can be considered with Taichi. Additionally, modifying images efficiently can be achieved through UE's own RenderTarget and Material.
Attentive developers may have noticed that TAPython does not provide a so-called get_image_data method for SImage. This is intentional because the SImages operated in TAPython are created by ourselves and the content is filled by us. Therefore, we also have a copy of the image content data in the Python end. Thus, "avoid repeatedly obtaining Image content from UI widgets and directly obtain it from Python data" is a more reasonable approach.
Event¶
The SImage widget supports the following three mouse events, making it easy for users to interact with SImage in the interface.
- OnMouseMove
- OnMouseEnter
- OnMouseLeave events
In the following example, the variable placeholders %uv and %mouse_flags will be replaced by actual uv values and mouse button flags at runtime.
{
"SImage": {
"DesiredSizeOverride": [200, 200],
"Aka": "ImageCanvas",
"OnMouseLeave": "your_tool.on_mouse_leave(%mouse_flags)",
"OnMouseMove": "your_tool.on_mouse_move(%uv, %mouse_flags)"
}
}
- %uv: tuple, (u, v)
- %mouse_flags: Flags. The 0th, 1st, and 2nd bits represent the left, middle, and right mouse button states, respectively. The values are bitwise OR'ed. For example, if the left and right mouse buttons are pressed at the same time, the value is 5 (1 + 4).
def on_mouse_move(self, uv, mouse_flags):
self.cursor_pos_x, self.cursor_pos_y = uv
self.bLMB = (mouse_flags % 2)
self.bMMB = (mouse_flags>>1 % 2)
self.bRMB = (mouse_flags>>2 % 2)
For example, we captured the user's mouse input in Image, and then passed it to StableFluid for calculation. For more details, see: Modify SImage content and Set Pixels to RenderTarget in Unreal Engine. The sample repository is at TAPython_Taichi_StableFluid_UE5.
