+ <div class="title-block">
+ <h3 class="blog-title">Let's get some images displayed</h3>
+ <h3 class="datestamp">20/03/2025</h3>
+ </div>
+ <p>
+ OK, it's 6pm, there's 3 hours until the Apprentice is on, let's
+ get hacking. The first thing I need to do is, uh... take the
+ thing apart.
+ </p>
+ <a href="../img/projects/pipboy/pipboyopen.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/pipboyopen.jpg"
+ alt="A photo of the pipboy's arse with a ribbon cable hanging out of it."
+ />
+ </a>
+ <p>
+ That's an micro-sd extender cable stringing out of it, there.
+ Nearly everything that makes up the pip-boy is stored on an sd
+ card which is, conveniently, not bolted into the board.
+ </p>
+ <p>
+ You can open up the thing and take it out, you can copy all of
+ its files over to a folder and, most usefully for us, you can
+ copy those same files over to another, bigger sd card (the one
+ installed is only 250MB) and it works, as long as the card is
+ fat32 formatted.
+ </p>
+ <p>
+ I've put a 32GB one in there, which is overkill, but I had it
+ lying around. It also means I can put the entirety of the
+ FO3/4/NV soundtracks on there, if I want. Which, maybe, I do in
+ future! Who knows.
+ </p>
+ <p>
+ More importantly, however, that SD card has a USER/ folder where
+ we can drop our own custom javascript files and it'll display
+ them in a nice 'APPS' menu in the INV menu.
+ </p>
+ <p>
+ We're gonna start with the the helloWorld and graphicsTest files
+ that are currently in the repo. Some file wiggling and inserting
+ and removal of SD cards and bang, there they are!
+ </p>
+ <a href="../img/projects/pipboy/pipboyapps.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/pipboyapps.jpg"
+ alt="A photo of the pipboy's arse with a ribbon cable hanging out of it."
+ />
+ </a>
+ <p>
+ So we want tio draw something to screen that isn't just text,
+ next. So I need to dive into some docs. More updates in a bit...
+ </p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <p>
+ OK, first thing I want to do is draw a square. Which means we
+ need to understand how the graphics buffer is working. See,
+ right here, in the dump of the buffer of the main portion of the
+ screen, is some interesting evidence.
+ </p>
+ <a href="../img/projects/pipboy/firmwarebCbuffer.png">
+ <img
+ class="blog-img"
+ src="../img/projects/pipboy/firmwarebCbuffer.png"
+ alt="A screenshot of an the graphics context in the pip boy firmware."
+ />
+ </a>
+ <p>
+ The interesting thing here is 'UInt8.' This is an array of 8bit
+ integers. This could mean the pixels are rendered as 3-byte RGB
+ values, with the r and b just ignored, or it could mean each bit
+ in the byte is a pixel, and the different tones is achieved
+ using dithering. Right now, I'm not sure!
+ </p>
+ <p>So, to find out, we're gonna draw three squares.</p>
+ <a href="../img/projects/pipboy/squares.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/squares.jpg"
+ alt="A photo of the pipboy screen showing 3 16x16 squares. One is slightly dimmer than the other two."
+ />
+ </a>
+ <p>
+ And there we have it! Three squares. Now if we look at the code
+ I've written, and note that the middle square is the dimmest of
+ the three, we can deduce...
+ </p>
+ <a href="../img/projects/pipboy/squarecode.png">
+ <img
+ class="blog-img"
+ src="../img/projects/pipboy/squarecode.png"
+ alt="A screenshot of the code to make the three squares. The three are commented 'full byte pp', 'dithered square', and 'fullbright square'"
+ />
+ </a>
+ <p>
+ That I was completely wrong and the screen is compeltely
+ monochrome, and any dimming is done by dithering. I'm so glad
+ I'm liveblogging this so everyone can see how stupid I am.
+ </p>
+ <p>
+ That's ok though! We're here to learn, and this actually makes
+ things relatively nice and easy. I'm sure there are monochrome
+ image -> uint arrays somewhere out there one the web already,
+ thisn is a very common format on embedded platforms, so I just
+ need to find one and run the icons for various perks etc through
+ them, and we can get one displaying.
+ </p>
+ <p>In fact, maybe that's what we do next. BRB...</p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <p>
+ Excellent, here's one, first hit on google.
+ <a href="https://javl.github.io/image2cpp/">image2cpp</a>
+ Let's run the Cherchez La Femme image through it and upload it
+ to the machine aaaand...
+ </p>
+ <a href="../img/projects/pipboy/errorlowmem.png">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/errorlowmem.png"
+ alt="A screenshot of the espruino terminal reading 'New interpreter error: LOW_MEMORY,MEMORY'"
+ />
+ </a>
+ <p>
+ So, that's not good. In fairness, I am loading a 167x167 bitmap
+ into memory here, I suppose it makes sense that that wouldn't
+ exactly work. But it is going to make this more difficult.
+ </p>
+ <p>
+ Next question then. What is a reasonable maximum array size we
+ can use here? Time to experiment. We'll start with half the
+ resolution, 83x83.
+ </p>
+ <a href="../img/projects/pipboy/chercheztake1.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/chercheztake1.jpg"
+ alt="A photo of the pipboy screen showing garbage in a square."
+ />
+ </a>
+ <p>
+ OK, it's displaying, but that's clearly not right. Here's what
+ it's supposed to look like:
+ </p>
+ <a href="../img/projects/pipboy/CherchezLaFemme.png">
+ <img
+ class="blog-img"
+ src="../img/projects/pipboy/CherchezLaFemme.png"
+ alt="The Cherchez La Femme perk icon from New Vegas"
+ />
+ </a>
+ <p>
+ That's ok though, there's options on the converter for just this
+ predicament. This was the default (Horizontal - 1bpp), and
+ Vertical - 1bpp landed similar results, so let's move on from
+ that converter and use the one hosted on
+ <a href="https://www.espruino.com/Image+Converter"
+ >Espruino's website</a
+ >
+ instead. Man I'm really proving myself soooo smart today.
+ </p>
+ <p>
+ Using the right tool for the job gives us this! And it's even
+ full resolution! Huzzah!
+ </p>
+ <a href="../img/projects/pipboy/cherchezsuccess.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/cherchezsuccess.jpg"
+ alt="A photo of the pipboy screen showing the Cherchez La Femme icon."
+ />
+ </a>
+ <p>
+ OK, that's real, genuine progress. We can encode images and
+ display them, which we need for the perks screen. They are
+ pretty big though, so I think next order of business is going to
+ be keeping those strings in text files and loading them in when
+ we need to. Back to the docs... Although I might leave it there
+ tonight, my back hurts. No wonder Zuck had to be 19 to do this
+ live.
+ </p>
+ <div class="title-block">
+ <h3 class="blog-title">Loading from files</h3>
+ <h3 class="datestamp">21/03/2025</h3>
+ </div>
+ <p>Evening! Let's get right back to work.</p>
+ <p>
+ Yesterday, we figured out how to correctly convert and display
+ an image on the screen. Today,we're going to figure out storing
+ those images, loading them at runtime, and switching to another
+ image later.
+ </p>
+ <p>
+ So, first up: storing and loading the image. This is gonna need
+ me to look up how the espruino fs library works. BRB...
+ </p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <a href="../img/projects/pipboy/imgstringdump.png">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/imgstringdump.png"
+ alt="A screenshot of the pipboys serial connection in vscode showing the file read from disk."
+ />
+ </a>
+ <p>
+ Well that was tremendously easy. First try baby, finally I'm not
+ completely stupid about something! We're gaming! I would take a
+ photo of the screen but I didn't change the image's position so
+ it looks exactly the same. but rest assured it is there!
+ </p>
+ <p>OK, next up, we swap images on the fly.</p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <video width="672" height="378" class="video-embed" controls>
+ <source
+ src="../img/projects/pipboy/pipboyperkswitch.mp4"
+ type="video/mp4"
+ />
+ Your browser does not support the video tag.
+ </video>
+ <p>
+ Done! Again! Wow we're going win after win today. Apart from the
+ weird bit of artifacting in the top left of the image there,
+ but, er... I'm sure that's fine. Let's ignore it for now. We'll
+ see if it continues with other images.
+ </p>
+ <p>
+ OK, so this is actually a fair cut through the work we need to
+ do here. So let's structure the screen a bit, and add the name
+ and descriptions.
+ </p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <p>
+ Hoooo boy I spent a while here huh! So much for 'live' blogging.
+ </p>
+ <a href="../img/projects/pipboy/perklist1.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/perklist1.jpg"
+ alt="A photo of the pipboy screen showing a decent approximation of the perk menu from FO3/NV. Action Girl is selected and shows its icon and description."
+ />
+ </a>
+ <p>
+ While I've been gone I basically drew the rest of the fucking
+ owl. Look! It's the screen from the game! Pretty much
+ completely!
+ </p>
+ <p>
+ There's some artistic license; in-game the perk description
+ displays in the same column as the image, but the available area
+ there is too small to display it readably on screen, so I've
+ bannered it at the bottom instead.
+ </p>
+ <p>
+ But yeah, we've got the basics of the screen here! The list of
+ perks, the box around the selected one, the image, the
+ description. They're all loaded dynamically from the list of
+ files on the SD card, and I've just gone in and tested the
+ reselection with a timeout, and hot damn, it works.
+ </p>
+ <a href="../img/projects/pipboy/perklist2.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/perklist2.jpg"
+ alt="Another photo of the pipboy showing the perk menu. Now Cherchez La femme, second in the list, is selected."
+ />
+ </a>
+ <p>
+ Additionally, if you're code digging, you'll see I've done a
+ bunch of reorganisation. While I was hacking before, I've gone
+ in and refactored and made all of this actually functionally
+ useful for building up the application proper.
+ </p>
+ <p>
+ Next thing, then, is input. Which hopefully, shouldn't be too
+ bad? I'll tackle that at some point over the weekend. Then it's
+ just filling the rest of the perk data (and fixing whatever is
+ wrong with the action girl image) and presto, that's a screen!
+ </p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <p>
+ OK I lied did a tiny bit more tonight. It's some housekeeping
+ code - I wanted to make sure the images displayed centrally if
+ they were cropped to just their actual data, any rows of empty
+ pixels around them removed.
+ </p>
+ <p>
+ This turned out to be a smart decision - It saves space, it
+ looks nice, and it meant I redid the action girl icon with its
+ bit of corrupted data, a bug I would have otherwise inevtiably
+ ignored until the end. No photos right now because not much has
+ actually changed but it's good and sets us up for just
+ implementing the manual selection now.
+ </p>
+ <div class="title-block">
+ <h3 class="blog-title">Chasing accuracy</h3>
+ <h3 class="datestamp">22/03/2025</h3>
+ </div>
+ <p>
+ Good morning! I'm up bright and early to continue my vital work.
+ </p>
+ <p>
+ Actually, I've already started. I looked up what the perk menu
+ looks like in Fallout 4 to check if there a) is one other than
+ the chart (there is) and b) if what I'm making is accurate to
+ that.
+ </p>
+ <p>
+ And it mostly is, but the selection box around the perk in the
+ list is actually all green and the text is black, see the below
+ image.
+ </p>
+ <a href="../img/projects/pipboy/pipboyfo4.png">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/pipboyfo4.png"
+ alt="A screenshot of Fallout 4 showing the pipboy open on the perks screen."
+ />
+ </a>
+ <p>
+ So I kinda want to recreate that. Doing so would be useful as
+ it'll also bring my selection menu in line with some of the
+ official submenus on the device as well, making the app look
+ more official :)
+ </p>
+ <p>
+ This is what I've got so far - I couldn't get the text to draw
+ in black so I settled on a halftone selection box instead. I'm
+ still not completely happy with it though and want to get as
+ close to accurate as I can, so I'm going to dive into the
+ firmware and see if I can divine how it does the black on white.
+ </p>
+ <a href="../img/projects/pipboy/halftoneselection.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/halftoneselection.jpg"
+ alt="A photo of the pipboy on the perks screen, showing the selected perk title with a grey box behind it."
+ />
+ </a>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <p>
+ Ooookay, a lot has happened since I said I'd check the firmware.
+ Here's the current state of the screen:
+ </p>
+ <a href="../img/projects/pipboy/accurateLayout.jpg">
+ <img
+ class="blog-img-lrg"
+ src="../img/projects/pipboy/accurateLayout.jpg"
+ alt="A photo of the pipboy on the perks screen. The perk title is now boxed with green pixels, with the text in black. The font of all text is also accurate to the game now."
+ />
+ </a>
+ <p>
+ So, like I said, I dove into the firmware code. I found quite a
+ lot of interesting stuff in there while trying to figure out how
+ they did the black on white textbox. Let's run through.
+ </p>
+ <p>
+ Top of the order - there are three foreground colours the pipboy
+ can draw in, 0-3. These are actually different brightnesses of
+ the pixels. So drawing a fullbright pixel is colour 3, an off
+ pixel is colour 0, but there's also colours 1 and 2 which are
+ slightly dimmer on colours!
+ </p>
+ <p>
+ This is how the fading 'Attachment' and 'Aid' labels at the top
+ are drawn, although it's not very clear in the photos I've
+ taken.
+ </p>
+ <p>
+ I actually discovered this by accident - I saw a setColor(0) in
+ the firmware and deduced that was black, so then I added that in
+ to my drawing code - but in so doing I also set the colour of
+ everything not black to setColor(1), which made everything
+ dimmer.
+ </p>
+ <p>
+ This was a bit of a eureka moment - I'd figured out that there
+ were roughly 4 colours displayed on screen but earlier assumed
+ this was all dithering after my experiments a few days ago.
+ Turns out I was right all along! Ha-HA!
+ </p>
+ <p>
+ So, using this new knowledge, I've added correct coloring to
+ everything. I've also chosen to dim the perk image a shade and I
+ think it makes it look really good. The full brightness was
+ overpowering some of the details of the icons.
+ </p>
+ <p>
+ While digging, I also found an interesting function call:
+ setFontMonofonto18(). This was a call on the graphgics context
+ and was pretty self-explanatory.
+ </p>
+ <p>
+ The thing is, I'd noticed my fonts were a bit off. They weren't
+ as tall as they should have been. Turns out, that's because
+ there's a custom font in use, but only when you specifically set
+ it. So because I was just using setFontVector() instead of these
+ Monofonto calls, the font was getting set back to the default
+ Espruino vector font.
+ </p>
+ <p>
+ Ctrl-F-ing 'Monofonto' in the firmware dump showed up a few
+ similar calls, listed here for convenience:
+ </p>
+ <ul>
+ <li>setFontMonofonto16</li>
+ <li>setFontMonofonto18</li>
+ <li>setFontMonofonto23</li>
+ <li>setFontMonofonto28</li>
+ <li>setFontMonofonto96</li>
+ <li>setFontMonofonto120</li>
+ </ul>
+ <p>
+ I assumed Monofonto was the font name, and tried to plug it in
+ to a few of the Espruino font functions, but got errors each
+ time, which was troubling. Giving it a quick duckduckgo
+ presented an excellent result:
+ <a href="https://www.dafont.com/monofonto.font"
+ >DaFont Monofonto.</a
+ >
+ Turns out the font used on the pip-boy (and presumably in the
+ actual game!) is freely available to download. Perfect!
+ </p>
+ <p>
+ With that in hand, I looked at the Espruino setCustomFont call,
+ and with a smidge more docs digging I found
+ <a href="https://www.espruino.com/Font+Converter">this page</a>
+ that converts a font and size into a graphics context function
+ call - just like the ones already in the firmware!
+ </p>
+ <p>
+ This was perfect. I plugged in the font and set size to 14,
+ dropped the code in my file, and now my description is in the
+ correct font. I used the pre-existing setFontMonofonto18 for the
+ Title - 16 was proving a little too small, and other menus in
+ the device are size 18.
+ </p>
+ <p>
+ I'm honestly over the moon about this. I had kind of already
+ settled for things not looking quite right, but with a bit of
+ digging, I solved basically all the imparities with the games.
+ </p>
+ <p>
+ What that isn't, though, is input. Which was the title of this
+ update (I've changed it now). So I'll tackle that next!
+ </p>
+ <div class="title-block">
+ <h3 class="blog-title">Actually doing input this time</h3>
+ <h3 class="datestamp">22/03/2025</h3>
+ </div>
+ <p>
+ OK we're back again baby. And this time I sweart I am going to
+ do input. To show willing I've even already started diving into
+ the firmware to work out how it's done!
+ </p>
+ <a href="../img/projects/pipboy/firmwareInputListener.png">
+ <img class="blog-img-lrg"
+ src="../img/projects/pipboy/firmwareInputListener.png" alt="A
+ screenshot of the dumped firmware. It notably shows a function
+ 'd' and a call Pip.on('knob1', d)")." />
+ </a>
+ <p>
+ So that firmware screenshot tells us all we need to know,
+ really. The Pip object has some events (in this case, 'knob1',
+ which is a) funny and b) the left hand wheel control) one which
+ you can register functions to call.
+ </p>
+ <p>
+ This screenshot is from the portion of the code that handles
+ switching between the different health animations, but this
+ applies anywhere really. So what we need to do is:
+ </p>
+ <ul>
+ <li>Create a listener and register it</li>
+ <li>
+ Make sure we play the knob click audio like the rest of the
+ device does
+ </li>
+ <li>Move our selection up and down the list with each click</li>
+ </ul>
+ <p>Pretty simple! OK, lets go do that.</p>
+ <b>Code Updated. Check the github link to keep up.</b>
+ <p>And look at that! We're done.</p>
+ <video width="672" height="378" class="video-embed" controls>
+ <source
+ src="../img/projects/pipboy/inputWorking.mp4"
+ type="video/mp4"
+ />
+ Your browser does not support the video tag.
+ </video>
+ <p>
+ Nothing really special to talk about here, just some basic
+ increment/decrement handling and looping back to the start of
+ the list when necessary.
+ </p>
+ <p>
+ There is a specific wrinkle of having to deregister the input
+ event. Early on in my testing I hadn't done that and it kept the
+ .js loaded even after removing the SD card, which meant it
+ looked like any edits I made weren't actually working. In order
+ to prevent this I added gracefulClose() which deregisters the
+ handler and shows the main menu again.
+ </p>
+ <p>
+ Last thing here is really to handle what happens when we have
+ more perks than will fit in our available space. Then this
+ screen is basically done!
+ </p>
projects / gnargle.github.io.git / blobdiff