Sketch to UI Design With AI: The 6-Step Workflow

A sketch is the fastest way to think and the slowest thing to hand over. Nobody can click a napkin. The gap between the drawing and something you can show a client used to be an afternoon in a vector tool — now it is a photograph, a sentence of instruction, and about a minute of waiting.

This is the working version of that process: what to draw, how to shoot it, what to type, and where the technique genuinely falls down. Every step below matches what actually happens when you attach an image to a prompt on this site.

What Image-to-Design Actually Does

Your sketch is not traced. It is read. The model looks at the photo as a description of intent — this is a header, that column of boxes is a card grid, the scribble in the corner is a logo — and then designs a finished, full-fidelity screen that follows that intent. The output is a rendered design, not a layered file.

That distinction decides everything else. A tracer rewards a neat drawing; an interpreter rewards a clear one. You are not trying to produce a beautiful sketch. You are trying to produce an unambiguous one, then let the prompt supply the taste.

Step 1: Draw for a Reader, Not for a Portfolio

Rough is fine. Ambiguous is not. Four habits carry most of the quality:

• Dark ink, pale paper. A felt-tip or whiteboard marker survives a phone camera; pencil on cream paper often does not.
• Label the zones. Write NAVBAR, HERO, 3 FEATURE CARDS, FOOTER CTA straight onto the boxes. Written words are read as instructions and are the single highest-leverage thing you can add.
• Cross out image areas. A box with an X through it is universally understood as a picture placeholder. An empty box is just an empty box.
• Show proportion, not decoration. Whether the hero is half the screen or a fifth of it matters. Whether your rectangle has rounded corners does not.

Skip the fine detail. Body copy is a few horizontal lines; a chart is a box marked LINE CHART. Detail you draw at pencil width will not survive the next step anyway.

Step 2: Photograph It Properly — This Is Where Quality Is Lost

Uploads are resized in your browser before they are sent: the long edge is capped at 1280 pixels, and anything that is not a PNG is re-encoded as JPEG at 85% quality. That happens for a practical reason — the reference travels inside the request as an inline data URL, and a raw 12 MB phone photo would not fit — but it has a real consequence for you. Fine pencil lines and small handwriting are exactly what a downscale destroys.

• Crop tight before uploading. If half the frame is your desk, half your 1280 pixels are spent on your desk. This is the single biggest improvement most people can make.
• Shoot straight on. A keystoned whiteboard photo reads as a trapezoid layout. Stand square to the surface rather than correcting it later.
• Kill the shadow. Side lighting turns a hand shadow into a dark panel the model may interpret as a design element.
• One screen per photo. Six sketches on one sheet become one crowded generation. Shoot and generate them separately.

Screenshots, exported Figma frames and tablet drawings all work the same way and start with a large advantage: they are already sharp, already cropped, and already square to the frame.

Step 3: Attach the Image and Write the Instruction

Drag the file onto the prompt box or use the attach button — on the home page, in the dashboard composer, or on any tool page. PNG, JPG, WEBP and SVG are accepted up to 15 MB, one reference image per generation. You will need a free account, because generating runs against your credit balance.

You can send a reference with no text at all, and you will get a reasonable interpretation. You will get a far better one by treating the sketch as the layout and the prompt as everything the sketch cannot say — style, palette, real copy, and the audience:

Use this sketch as the layout for a project-management app landing page.
Boxes with an X are product screenshots. Keep the section order exactly.
Style: light, restrained, one indigo accent, generous whitespace.
Headline: "Ship it before Friday". Button: "Start free".
      

Two lines are doing quiet heavy lifting there. Telling the model what your placeholder marks mean removes the one ambiguity a sketch cannot resolve on its own, and quoting the copy is what stops invented lorem-ipsum-flavoured text appearing in your headline. Anything you do not put in quotation marks gets written for you.

Step 4: Choose the Format Before You Generate

The format sets the canvas ratio, and a design generated at the wrong ratio has to be redone rather than cropped. Pick it to match where the design will actually live:

Step 5: Refine by Region Instead of Rerolling

The first generation is a draft. The instinct is to regenerate until something lands, which throws away the eighty per cent that already worked and costs credits each time. The better move is to correct the part that is wrong.

In the editor, drag the region tool around one element — a button, a card, the header — and describe only that change. The selected crop and the current design are both sent along with your instruction, so the model edits in place rather than starting over:

Make this card the same height as the two beside it and move the
price above the description. Keep the palette and the icon.
      

Two or three changes per pass works well; ten competing instructions in one sentence does not. And if a whole section came out wrong, it is usually faster to fix the sketch label that caused it and generate once more than to argue with the output.

Step 6: Export

Finished designs export as PNG or JPG at 1x, 2x or 3x. Use 2x or 3x PNG for anything going into a deck, a client review or print, and JPG only when file size genuinely matters. There is no watermark on any export, including on the free plan — what you see in the editor is what lands in your downloads folder.

Three Prompt Recipes for Sketch Conversion

Four Mistakes That Waste a Generation

• Uploading the sketch with no instruction. The image gives layout. Without a prompt, style, palette and every word of copy are guesses.
• Photographing the whole page. The long edge is capped at 1280 pixels; margin you photograph is resolution your sketch does not get.
• Expecting a traced copy. The sketch is read as intent, not geometry. If a proportion is non-negotiable, write it in words as well as drawing it.
• Regenerating instead of region-editing. Rerolling discards what worked and spends credits doing it. Correct the one element that is wrong.

Frequently Asked Questions

Can AI turn a hand-drawn sketch into a UI design?

Yes. Photograph the sketch, attach it to a prompt, and the model reads your boxes and labels as layout intent, then designs a finished screen around them. Quality tracks the clarity of the drawing and the specificity of the prompt far more than it tracks how neat your handwriting is.

How do I prompt AI for UI design from an image?

Let the image carry the layout and let the prompt carry everything it cannot: the platform, the visual style, the exact palette, what your placeholder marks stand for, and the real copy in quotation marks. Two to four sentences is usually the sweet spot. There is a longer breakdown in our guide to writing AI prompts for UI design.

Is sketch to UI design AI free?

You can start free with an account — generating uses credits from your balance, and every export is watermark-free, including on the free plan. There is no separate charge for attaching a reference image; a generation from a sketch costs the same as a generation from text.

What image formats and sizes can I upload?

PNG, JPG, WEBP and SVG, up to 15 MB, one reference image per generation. Larger images are downscaled in the browser to a 1280-pixel long edge before sending, so crop tightly rather than uploading a wide shot of your desk.

Can I export the result to Figma or to code?

No. Designs export as PNG or JPG at 1x, 2x or 3x, not as Figma layers or HTML. If your workflow needs editable vectors or a component tree, use this stage for exploration and direction — settling the layout and the look in minutes — then rebuild the chosen direction in your design or code tool. If editable layers are the deliverable itself, a Figma-export tool is the better fit.

Why does the output not match my sketch exactly?

Because it is an interpretation rather than a trace. Elements get proportioned for balance, and anything ambiguous in the drawing is resolved by the model. Tighten it by labelling zones on the sketch, restating critical proportions in the prompt, and then correcting the remaining differences with the region tool instead of regenerating.

Can I upload several sketches for a multi-screen flow?

One reference image goes with each generation, so run a flow one screen at a time. Generate the first screen, get it right, then generate the next with its own sketch and a line asking for the same style, palette and type scale. Consistency comes from repeating the style sentence, not from batching the uploads.

Does it work with Figma wireframes and screenshots?

Yes, and usually better than with paper. An exported frame or a screenshot is already sharp, cropped and square to the frame, so nothing is lost to the downscale. Just be explicit about what you are borrowing — a screenshot used for layout and a screenshot used for visual style need different instructions.

Which is better, a sketch or a text prompt?

Use a sketch when the arrangement is the thing you care about and describing it in words would take a paragraph. Use text alone when you know the style and content but have no strong opinion about layout. Attaching a sketch and writing a specific prompt is better than either, because the two supply different halves of the brief.

Sketching stays the fastest way to think through an interface. What has changed is that the drawing is no longer where the work stops — photograph it, describe what it cannot say, and you have something to put in front of someone before the idea goes cold. Browse the design tools to start from the format you need.