Problem with generating pixel representation of image upload in Adobe XD

The file of the rendition itself has the read method.

In your code:
results[0].outputFile has the read method. so It would be like:

const arraybuffer = results[0].outputFile.read({ format: require(“uxp”).storage.formats.binary });

could you post an example that uses the js-base64 library?

I tried it this way. Creating the rendition and everything works but not creating the arraybuffer. It’s empty.
Is there an update coming out which allows to read out the pixel values of an image easier?
That seems quite complex just for getting some pixel values. An operation which I think many Adobe XD plugins will need one day.

if you console.log it, it will look like an empty object {}, but it’s not actually an empty object. Try converting it to base64 and log it out, or send it to your server and see.

We’re aware of the difficulties in generating pixel representations; we’re working on improvements here (unfortunately I can’t provide a timeline yet).

But for now, this is what we have to work with, for better or worse.

Turns out that was a bad recommendation. I ended up using the npm library base64-arraybuffer, which works great.

I posted an example at Please include base-64 utilities.

Can you give an example of how to iterate through image pixels with the base64 encoded string? Everything works fine for me until the point were I have to use the Base64 string to get out the single pixel values. Creating the image rendition and converting encoding it to Base64 worked!

Can you please clarify what you mean by “get out the single pixel values”?

I need a pixel representation of the selected image. My plugin needs to know every rgb value of any pixel in the image in order to do its processing. Since one can’t get this representation out of the ImageFill object your API is offering I followed the steps in this channel and created an Image Rendition and with outputFile.read I generated an arraybuffer. After that I used a base64 library to convert it to base64 as you suggested. The problem I have now is how to get the single pixel values? I don’t really get the connection between base64 and the image pixel representation :confused:

Shall I add more details to my problem?

I am not totally aware of how there is a connection either. I re-read the thread and it seems like this method was suggested when you asked about getting the arraybuffer. Unfortunately, we don’t have any direct API that produces rgb values of an image. I was assuming you would be able to get what you need if arraybuffer is provided.

Does anyone else have any suggestions here?

@stevekwak Your colleague kerrishotts made the initial suggestion with the ArrayBuffer. @kerrishotts Is there a way to access the image as pixel representation and do some analysis with for instance the rgb distribution? Using the Arraybuffer ? My Plugin is generating a Color Palette with the most prominent colours in an image and I need the values of every single pixel.
The problem is I don’t get the connection between arraybuffer and pixel representation.

Since no one seems to care anymore I will look for other Platforms to integrate my plugin. Thanks for the former responses though. That counts also for all my planned future plugins. I will try at Figma…

Would a library like https://www.npmjs.com/package/pngjs help?

The ArrayBuffer will contain binary data encoded in the particular format you exported the rendition to (e.g. JPG, PNG) - so you need to find a suitable JS library to extract the info from that format. Seems like the library above can give you r,g,b,a for each pixel value of a PNG, provided you can require and run it in the XD environment.

1 Like

@stavros Don’t get mad if you don’t get help. The Adobe folks here are helpful but busy, and others try to help but are also completely consumed with our own work, and generally can only help if your question is something they’ve already encountered and can answer fairly simply.

I agree, though, that there should be a simple way to get the bits from an ImageFill as an arrayBuffer (just like there should be a way to turn arrayBuffers into an ImageFill without going through base64 encoding/decoding), with enough information to interpret the arrayBuffer structure.

1 Like

Sorry I overreacted. I’m under some pressure to release this plugin and shouldn’t have responded that way. I’m thankful for your help!

No problem–just didn’t want you to think people aren’t trying…

XD extensibility is pretty young still, and there are lots of missing pieces. I’d give Adobe time, though–they’re moving very quickly in all the right directions.

1 Like

Yes people are trying. I just somehow had the impression I was being ignored. It’s rather that people are busy themselves.

1 Like

Apologies for the delay, @Stavros… the same Adobe people who participate in this forum are the same people building XD and UXP, so there’s not always time in our day to craft a response. Apologies, though, that you felt ignored – that’s certainly not the intent!

For working with the actual pixels – there’s no easy way, unfortunately. IIRC, plugins that have done that will use an npm package that can parse the PNG or JPG file format and then operate on the pixels that way. You might try this one: https://www.npmjs.com/package/pngparse

We’re working on improving this in upcoming releases, but for now, the easy answer is that there is no easy answer.

1 Like

@stevekwak I somehow run into the error :

const arraybuffer = await erg[0].outputFile.read({ format
: require('uxp').storage.formats.binary});
                        ^
Plugin SyntaxError: await is only valid in async function

Am I missing something ? Every method has the await keyword in.