Oldskooler Ramblings

…is not any of their keyboards (although I have very fond memories of playing with my Casio VL-Tone and SK-1). It’s their first digital camera; in fact, the first consumer 3MP digital camera, the QV-3000 EX.

Mine broke a while ago. I finally found a replacement at a reasonable price (ie. under $20). I am taking pictures again. I am happy.

Why so happy over a dumb 3MP consumer camera? Because I am not a professional photographer and I don’t take pro pictures. The QV-3000 EX has just enough control for a noob like me to take manual-focus close-ups:

…without looking like a complete beginner. And it’s night mode surprised the hell out of me:

That picture was taken at midnight with the flash and the two house lights providing the only illumination. I wasn’t even using a tripod and it’s not blurry! That’s insane.

Through the help of two good friends, I am now back on the grid, and (cue insidious music) stronger than before.  I finally have a dual-core rig, for one thing.

I have enough parts left over to construct a second machine to replace my ailing fileserver, so that’s my next project.  But before I start that, I’m going to write a curious new utility that will help anyone with digital picture frames.  More later.

I was first made aware of Net Op Systems (currently going by the name of NOS Microsystems Ltd.) when downloading Adobe Acrobat Reader about 3 years ago. I was struck by how small the compressed deliverable was, so, being a compression hobbyist, I did some preliminary analysis and found that they used a considerable about of context arrangement and prediction (ie. “solid” mode in rar/7-zip, or the content-specific predictors in PAQ) to get the size down. I recently ran across their product again when downloading the most recent version of Solaris x86; it comes in a 1.1G NOSSO executable package. The sole payload was a 3.1G .ISO image, which meant the compressed deliverable was 37% of the uncompressed size.  This is very impressive, given that the .ISO image is filled with a lot of .JAR and .BZ2 compressed images themselves. The successful extraction of a workable .ISO file from this compressed deliverable means that NOSSO has to perform the following to work its magic:

1. Identify the various compressed files in the .ISO wrapper
2. Extract those compressed files
3. Decompress the content inside those compressed files
4. Arrange everything by context (ie. all ASCII text in one group, all binary executables in another group, etc)
5. Compress the entire thing to a proprietary stream, using content-specific prediction for various content groups
6. Store the original arrangement of the compressed and uncompressed content

Upon extraction, the NOSSO distributable has to perform the following:

1. Decompress everything and keep track of it
2. RECOMPRESS the data originally found in compressed files, so that their effective format is kept the same. There may be small differences due to compression options and implementations, but as long as the end result is usable by the end program (ie. a reassembled .ZIP file is still able to be decompressed into the same contents) then there’s no harm done.
3. Rearrange the end result back into the original container (in my case an .ISO file)

This is why they call their process “reconstitution” instead of “decompression”, because the end result, while functionally identical, is usually not bit-for-bit identical. By taking advantage of context and recompressing files from less-efficient formats into the more efficient format NOSSO uses internally, we can get these excellent compression ratios. (In fact, I’ll wager that, to speed up reconstitution times, they use a very fast and less efficient version of recompression of the files inside the target wrapper, which would inflate them slightly and result in even more “impressive” compression ratios :-)

What’s the downside? The downside is that this entire process defeats its own purpose. I’ll explain:

NOSSO is marketed as a delivery format that saves everybody bandwidth and, presumably, time. It’s that presumption that allows them to shoot themselves in the foot. While the compressed distributable only took 39 minutes to download on my 6mbit/s cable modem connection, it took a whopping 124 minutes to “reconstitute” on a 2.6GHz P4 with 700MB RAM free (out of 1G RAM total). My total time to get the end result was 163 minutes. (A 2.6GHz machine is not the bleeding edge in 2007, but it’s no slouch either, and is representative of the average system most people will use for everyday use.) At its original size, 3.1G, it would have taken me only 104 minutes to download it.

It would have been faster to get the end result had it not been compressed at all.

Now, 6mbit/s is a pretty fast broadband connection, so I understand that skews the results a bit. With a more common broadband connection speed of 3mbit/s, let’s check the numbers again: Compressed download + extraction: 202 minutes. Uncompressed download: 208 minutes. Okay, so it’s break-even at a 3mbit/s connection. But break-even still involves 100% CPU utilization as the thing is decompressed, resulting in an unusable system for two hours, so it’s still not “free”.

Is there strength in using any compression at all? Let’s check both WinRAR and 7-Zip on the original 3.1G unmodified .ISO file:

• 7-Zip compressed size: 2.68G. Time to download at 3mbit/s: 187 minutes. Decompression time: 14 minutes. Total time to get the end result: 201 minutes.
• WinRAR compressed size: 2.69G. Time to download at 3mbit/s: 189 minutes. Decompression time: 3 minutes. Total time to get the end result: 192 minutes.

So, at 3mb/s, the end result was just about the same, except our system was only tied up for 3 or 15 minutes instead of two hours. We’d get even more compression at the same decompression speed if we burst the .ISO like NOSSO does, compressed using WinRAR’s or 7-Zip’s “solid” mode, and then reconstitute it back into an .ISO when done with a small utility program.

My conclusion from all this is that there’s really no point in using NetOpSystem’s product, unless the end-user’s broadband speed is 1mbit/s or slower. But if it’s that slow, the user is already used to ordering DVD-ROMs for delivery instead of trying to download them, right? Or, if the user downloads them anyway, they’re used to firing them off before they go to bed, to download overnight. So, again, no need for the product…

…unless you’re the content producer and want to transfer cost (bandwidth) to the end user (time). Which is probably why NetOpSystems is still in business.

My anality reared its ugly head — I couldn’t take this computer-in-unknown-state any longer, so I gathered up my birthday money and bought a new power supply and motherboard. Eight hours later at 4am, I finally heard my first BEEP — one long and two short, which was a memory problem. I can’t tell you how wonderful it was to hear the damn thing finally speak to me. Some DDR swaps, and we finally boot.

So the motherboard was toast. Problem solved. So why the new power supply? Because it didn’t quite pass the ATX power supply tester I have. You see, the tester is supposed to show lights and beep. The one I was given showed lights… but no beep. So it was flaky. New motherboard: $70. New power supply: $40.

Total cost of this free machine: free $50 $90 $160. And I still don’t have a PCI-e video card yet (my old Diamond Viper is currently driving the machine. Yes, that’s right kids, a TNT card from 1998 is sitting in an Athlon 64.)

The best part? I scraped off the thermal goo and found it’s an Athlon 64 3200 — as in, the original Athlon 64. It’s slower than my main desktop machine (an Athlon XP). I have now paid upwards of $160 for a machine that is effectively three years old and slower than what I already have. Viva l’irony!

Don’t get me wrong; I think the entire process has been a hoot, and I love solving PC hardware problems. Better yet, the motherboard supports Athlon X2, so when I scrape together some more cash (probably my next birthday ;-) I can swap in an X2 and finally join the world of multi-core computing. It’s all good.

In sympathy to my financial plight, and possibly as a birthday present (I’m 36 today, whoopidy-doo), a coworker donated an Athlon 64 rig to me so that Sam could have a fast enough machine to run voice recognition software so he could email his family. (Sam has processing delays akin to autism and has trouble typing quickly.) I received a case, power supply, unknown Athlon 64 CPU hidden under a giant Zalman heatsink, motherboard, and RAM — which is more than powerful enough for voice recognition. All I need to do is add some spare hard drives lying around and it’s functional. It was a very generous donation.

It also doesn’t work.

It won’t POST, other than the fans spinning up; that’s it. No beeps, nothing. So, here begins the shopping list I’ve had to purchase in order to troubleshoot the thing:

• 20-to-24-pin power supply adapter: $4

…because the PSU was 20-pin and the motherboard was 24-pin. No change in operation. Now we test the power supply itself:

• Power supply tester for ATX 2.0 power supplies: $20

…confirmed that the power supply is functional. Moving on:

• Motherboard POST code display and PCI power testing board: $26 (cheap!)

…which I’m still waiting to arrive in the mail. This will let me see any POST codes as well as making sure power is going through the motherboard and hitting the PCI slots. If it starts to POST but stops, then probably the CPU is bad. If it never POSTS a single code, the motherboard is bad. Either way, this “free” gift has cost a minimum of $50, with at least another $80 (CPU or motherboard, whichever I find is bad).

Still, the thought was nice. And heck, getting an Athlon 64 rig for free $50 $130 is a great deal.

I’m not down or anything, I just find the situation ironic (and, by proxy, funny).

After nearly two weeks of unsuccessfully trying to restore the HighPoint RAID mystery sector information that ties my partitions together in a RAID 0 bond of harmony, I’m going to chuck it (the RAID controller, not the computer).  Serves me right for trying to do RAID on the cheap.  The new plan is to attach the drives as Plain Old Dumb Drives(tm) to the PATA ports, restore the XP partitions to a PlainOldDumbDrive, hope it works, and then fire up XP using any means necessary just long enough to export all proprietary databases to open ones (iTunes ratings, Thunderbird filters/junk/prefs, etc.).  Then I most certainly will be rebuilding the machine.

You know, I’ve had this XP partition/setup since 2001?  I guess that’s a testament to the stability of XP.  If you don’t act like a fucktard, XP won’t act like a fucktard back at ya.  I’ve had nearly no issues with XP in 6 straight years, which was a refreshing change from Windows 9x.  What prompted me to purchase Windows XP in 2001 was an experience in Windows 98 that almost had me damaging equipment:  I lost the ability to drag icons.  That wasn’t what made me mad, though: What made me furious was that, a week later, it fixed itself and started working again.  That’s just retarded.

So anyway… the new approach in trying to manage all this is System Rescue CD, brought to you by the fine folks who created partimage.  It’s a Linux rescue CD (a “liveCD” that you can boot directly and use without installing Linux) that works pretty damn well.  It’s taken Linux a long time to figure out how to do NTFS properly, but sysrescuecd distribution works well enough that you can mount a SAMBA/Windows share to a local directory and back up entire partitions to it.  I’m in it right now (running FireFox to add this weblog entry, no less).  In one window I’m copying my Acronis backup images to one of the PlainOldDumbDrives (so that multiple restores will take 30 minutes instead of 5 hours); in the other window I’m trying to figure out why I’m getting only 50mbit/s over my 100mbit/s FD network.  It’s plain, but functional, and can even boot off of a 256MB USB key.

All this because Apple couldn’t keep their iTunes database in XML by default.  Or, making even more sense, store the rating information in the MP3 files themselves using ID3 tags like everyone else.  I was lured into a false sense of security when I learned OS X was based on BSD Unix.  I thought maybe, just maybe, Apple would play nice with the rest of the world for a change.  Silly Trixter!

I’m not giving up yet.  I’m going to get my 14 months of song ratings back if it nearly kills me! Until then, I get to irritate my friends and family by posting blog entries from public computers while my email goes unread for, oh, going on 16 days now.

By this point I’m sure my friends and family are urging me to just forget trying to restore my old machine and just reinstall and start over.  I’m still working on restoration.  Why?

Because I spent over a year rating much of my 10,000 song library in itunes, and that information is not properly backed up.  I figure 2+ weeks of lost time is worth it to save 50+ weeks.

Where have I been lately?  Off the grid.  What originally started as a personal decision has now turned into a technical necessity:  The hard disks in my main machine have blown and we’re broke until I get paid next month.  I know I have other machines, but I’ve used this one machine for all of my main communication (email, IM, etc.) that it doesn’t feel “right” working on another machine for my day-to-day stuff.  Call me anal.

Next month’s payday is, conveniently, my birthday, so I will reward myself with two brand new hard drives.  Nothing fancy; I’ll probably do what I always do when buying hard drives:

1. Go to newegg.com
2. Look at every single hard drive with the specs I’m interested in; for example:
   • SATA or PATA
   • 3  year warranty at bare minimum
   • etc.
3. Divide the cost of the drive by # of gigabytes to get cost per gigabyte

…and pick the drive with the best value.

Why two drives?  Because I hate losing data due to circumstances beyond my control.  Must… mirror… data…

I had hoped to completely update the 8088 Corruption webpages before posting this, but it’s going to be at least another week and people have been asking me for it, so: An edited video of my NOTACON/Block Party 8088 Corruption Explained talk is available at archive.org. All of the embarrassing and missing parts have been fixed, added, edited, massaged, spindled, and mutilated, and it should be completely watchable. I replaced most of the bad video-camera-aimed-at-the-monitor footage with the actual conversion footage, filled in the hey-where’d-my-electricity-go? missing section with a voiceover, replaced all filmed slides with the actual slides, and took out two embarrassing swears (embarrassing not because they were swear words, but because I was nervous and stumbled over them).

While it is tempting to watch the flash version in a browser, I went through a great deal of trouble to make the MPEG-2 version perfect, including true 60Hz video in places. If you can spare the time, grab the MPEG-2 version and watch it on a real set-top dvd player for full effect. (Or a software player that isn’t broken; for example, use my favorite MPEG-2 player, VLC, with Deinterlace set to Linear.)

Work and home have been particularly busy this week and will be next week, so I apologize in advance for not having the extra movies, updated 8088 player, full source code, etc. available on the website yet. When I do, I’ll make a note of it in this blog.

I realized today that I’ve been very lucky when it comes to technology and surprises. Previously I mentioned the surprise IBM 5153 I found in storage, but today I was able to count many more:

• An i-river iFP-380 128MB MP3 player. Shortly after purchasing it, some “beta” firmware surfaced on the official support website that turned the player into a USB storage device. This means you could copy music onto and off of the device without using the severely idiotic crippled software that came with it. To date, I think this is the only hardware player series they’ve produced that allows you to do this — all prior and later models require the software to enforce DRM.
• My ReplayTV 5040. It was a Christmas gift, but shortly after I got it, DVArchive was released, and allowed me to suck the shows off of the unit via ethernet via a sensible GUI. This greatly increased the personal value of the unit, as I like to archive shows that are important to me.
• In the early part of the new century, I purchased a 3ware Escalade 6400 RAID-5 controller, intending to put it into a Windows system to function purely as a file server. A few days after I purchased it, and before I opened the box, 3ware announced Linux kernel driver source for driving the card, and a GUI to manage the card remotely! Needless to say, I installed Linux on the machine (a P933) and it’s been running for nearly 7 years — and does much more than a typical file server would, thanks to Linux.
• In a similar nod to the PC/XT I found, I pulled an IBM PCjr out of storage and tested it — and found out it was much more than I realized it was. When running Flight Simulator side-by-side with said XT, it was running faster than the XT! Further investigation showed that it is probably the most souped-up PCjr I will run across: It has a Jr HotShot installed (brings the machine to 640KB without additional memory sidecars), the 8088 was replaced with an NEC V20 for an additional 50% speed boost, and the motherboard was rewired slightly with the Tandy 1000 graphics modification (allows Tandy 1000 programs to run without graphics corruption). Oh, and did I mention that the packing material used in the box was a complete PCjr Newsletter set?  Dang!
  

Anything like this ever happen to you? Technology goes from good to awesome as a complete surprise?