One of the responsibilities of the Technical Coordinator in the Ohio Section is to submit something for the Section Journal. The Section Journal covers Amateur Radio related things happening in and around the ARRL Ohio Section. It is published by the Section Manager Tom – WB8LCD and articles are submitted by cabinet members.
Once my article is published in the Journal, I will also make it available on my site with a link to the published edition.
You can receive the Journal and other Ohio Section news by joining the mailing list Tom has setup. You do not need to be a member of the ARRL, Ohio Section, or even a ham to join the mailing list. Please sign up!
If you are an ARRL member and reside in the Ohio Section, update your mailing preferences to receive Ohio Section news in your inbox. Those residing outside the section will need to use the mailing list link above.
Updating your ARRL profile will deliver news from the section where you reside (if the leadership chooses to use this method).
Go to www.arrl.org and logon.
Click Edit your Profile.
You will be taken to the Edit Your Profile page. On the first tab Edit Info, verify your Email address is correct.
Click the Edit Email Subscriptions tab.
Check the News and information from your Division Director and Section Manager box.
Click Save.
Now without further ado…
Read the full edition at:
THE TECHNICAL COORDINATOR
Jeff Kopcak – TC
k8jtk@arrl.net
Hey gang,
Last couple times, I’ve been talking about my journey to preserve legacy media. First talked about different media formats and last month described how to create and use floppy disk images. This month is about optical discs, copy protection and storing images for preservation.
Optical disc images
Unlike floppy images, creating and mounting optical disc images was a hole other ball game. CDs have a variety of structures: data only (Digital Data), audio only (Digital Audio), CD-TEXT (artist and song details for Digital Audio), mixed mode (data on track 1, audio on tracks 2 – n), Enhanced CD (audio for audio players, data and multimedia for computers), and multi-session (data added or modified over subsequent writes to the disc). There are other standards such as MP3 CD, video CD, super video CD – those are all data tracks. DVD and Blu-ray are also data tracks.
Creating optical disc images
To meet my goal of having a raw data dump of optical media, Linux had the hardest time creating images. cdrao (CD recorder disc-at-once) can process different disc structures but the output files were not in a format most tools understand. Popular K3B disc writing program cannot make images of audio or mixed mode discs. dd won’t work either because it uses file structures (FAT, NTFS, Ext, CDFS, UDF). CD-audio is audio bits containing no file structure. Single track, digital data only discs were fine for Linux tools. Mixed mode and audio discs were a no-go.
On Windows, my long-time favorite, ImgBurn (freeware) made it all happen using the “Create image file from disc” option. Though not updated in some time, it still works well and the developer answers questions in the forum. It handled everything I threw at it. It defaults to ISO. If the disc doesn’t meet ISO9660 specifications, it creates the more flexible BIN & CUE formats. BIN file is binary data from the disc. CUE, for cue sheet, it a plain text file containing details about the tracks in the BIN file. This paring would be for CD-TEXT, multi-session, mixed mode, and Enhanced CD discs. UltraISO (trialware) created these images too and could output to different formats: ISO, Bin/Cue, Nero, Compressed, Alcohol, and CloneCD.
Copy protected discs
Then things went downhill quickly. One exception to “everything I threw at it” was copy protected discs. Copy protected discs aims to confuse the drive’s read system. A full read of a copy protected disc will fail. However, when activated, the protection software knows what to expect based on how it instructs the drive to read the disc. Many of these schemes are explained in this article. Other copy protection schemes install malware, called RootKits, that hides activities making detection and removal of the malware nearly impossible. Sony/BMG Music got caught installing RootKits in 2005. A user simply inserting the audio disc into their PC would unknowingly infect their system. As it turned out, companies were more concerned about their intellectual property and less about making software that didn’t have vulnerabilities. In the end, copy protection only hurts those who follow the rules.
I had one of those Sony/BMG discs. When I realized what they were doing, it was promptly returned. When referring to copy protected discs, I’m referring to a handful of unreadable game discs I have. Programs out there like Alcohol 120% (paid version) make perfect 1-to-1 copies, emulating copy protection schemes. It has been 15 years or more since I used those features but it worked great back then.
Failures creating working images using Alcohol 120% and CloneCD (trialware), which still tout making perfect 1-to-1 disc copies, I though was an issue with the application. After digging at the problem, I learned it’s probably not the fault of the application at all. First, I would identify discs with copy protection as ones ImgBurn showed had read errors. Next, make a new image of the disc using a 1-to-1 copy program. Then validate the image by installing the game on my Windows 7 64-bit operating system I was using to preserve legacy media. Finally, seeing if the game would run successfully. None of those games would launch. I spent waaaayyy too much time working under the presumption the problem was creating a good image. In reality, none (I mean NONE) of the copy protected games would run using their original discs. Imagine that, copy protection that doesn’t work.
Reasons copy protection wouldn’t validate successfully could be any of: a newer OS. These games are from the Windows 98/2000/XP era and cannot run on Windows 7. Running a 64-bit operating system when the copy protection drivers were written for 16 or 32-bit OSes. Could also be proactive blocking of the driver by Windows or Microsoft Security Essentials. With that information, though, I cannot say if those images created do or do not work. I would have to go down the road of getting an older operating system up-and-running. Could fire-up a Virtual Machine as well. I’ll pursue that later. Reading up on making images of copy protected discs, a disc drive that can read raw data is needed. While most noted drives state they read raw data, they really cannot. I couldn’t find a list of known working CD/DVD drives.
Avenues I looked into are sites that have cracks to bypass the copy protection validation schemes such as GameBurnWorld or GameCopyWorld. Not responsible for any damage or legal issues. This is for informational purposes only. Some cracks that I tried were for 16-bit OSes which is just not supported in a 64-bit OS. While I’m sure most of these games are available on a modern platform like Steam, I’m not feeling charitable enough to hand over more money to them seeing as they got it wrong the first time. Microsoft thinks Windows Compatibility Mode will fix all the problems. I think it only works on 32-bit versions of Windows. Most PCs are 64-bit, and 32-bit OS support is being dropped. I’ve never gotten any 16-bit Windows program to work in compatibility mode on a 64-bit OS.
Need to make copies of protected DVD or Blu-rays? See the products list at the CloneCD link above. Not responsible for any damage or legal issues. This is for informational purposes only. Unlike the game copy protection schemes which require software or a driver on the PC, DVD and Blu-ray store encryption keys on the disc which makes it fairly easy for programs like AnyDVD or DVDFab to read disc level encryption.
Mounting and using optical disc images
Things didn’t get much better when mounting disc images using virtual drives. Much like floppy disk mounting programs, I wanted something to emulate a CD drive on the host operating system. All programs I tried mounted ISO images to the operating system: Virtual CloneDrive (freeware), ImDisk (open source), Alcohol 120% (free edition), Daemon-Tools Lite (though installer is very bloated with crapware and maybe DNSBL on PiHole), UltraISO, and WinCDEmu (open source). Few of those programs mounted BIN & CUE correctly and even fewer handled multi-session images. Not all virtual drives are created equal. It may take some time to find a program or combination that works. In Linux, ISOs could be mounted using the “Disk Image Mounter” in the desktop GUI or using the command line (see part 2 in this series). Mounting BIN & CUE files in Linux required CDemu (open source).
Audio BIN & CUE files could only be mounted using Alcohol 120%, Daemon-Tools, and CDemu. An audio player like VLC (open source) would be used to play audio tracks. Foobar2000 (freeware) can play BIN & CUE files directly (without mounting). Enhanced CD and multi-session CD data tracks could not be accessed when mounted through any of the virtual drive applications I tried. Once the virtual drive hits the first lead-out in the image, that’s it. This affects images where data tracks follow audio tracks and multiple session images containing more than a single data track. I was never a fan of creating multi-session discs but I did have discs from friends that were.
UltraISO can access those data tracks from multi-session images and extract files. Maybe easier to copy the files from the disc to a folder instead of making an image for simplicity. There were two ISO editors listed for Linux, neither listed BIN & CUE file support. For completeness, all disc and structure data are still stored in the BIN image file and described in the cue sheet. It is a shortcoming of these virtual drive applications to not provide access to all data contained within the image. I have no idea why. Taking the same image and re-writing (burning) it to a blank disc would result in a complete copy of all sessions and data.
If possible, through the mounting software, mount the image READ ONLY! (see reasons in earlier parts). In addition, many virtualization and hypervisors such as DOS-box, VirtualBox, and vSphere can mount images naively to a guest operating system. Wikipedia has a comparison of disc image software applications for other suggestions.
Storing images
Lastly on this charade, storing these image files so they may live on forever! CD and DVD images are going to take up more disk space because the media can hold more data. Organize all images into a folder structure that makes sense: games, types of games, graphics, amateur radio, audio/video programs, operating systems, utilities, etc. I decided to store these images on my Network Attached Storage (NAS) with copies both off-line and off-site. The NAS file share is set for read-only to protect unintentional modification or deletion of the images or its contents.
Hard drives, until the beginning of this year, were relatively inexpensive. A 4-terabyte drive can still be purchased for around $100. Higher-capacity drives have been met with shortages and prices to reflect their supply. 4 TB is ALOT of storage. Use a new dedicated drive for storage, keep them on a local hard drive, or use an external hard drive. Make copies onto separate hard drives, USB thumb drives, or in “cloud” storage.
Plan a backup strategy sooner than later. The following is true for ANY data: data does not exist if it is not in two separate places. I argue three copies of data or it doesn’t exist (see #5 under “What can I do to protect myself?”): 3 copies of your data, 2 of them on different media (spinning hard-drive, sold-state SSD, thumb drive, optical, in the cloud, etc.), 1 must be off-site (at work, at a friend’s, storage locker – preferably temperature controlled/waterproof, safety deposit, with a relative, in the cloud).
Going more technical and into file system technology, use a file system that hashes files such as Btrfs or ZFS. Then scrub the data every couple-to 6 months. This keeps data in-check and detects errors in storage indicating media degradation or imminent failure. Linux has these features as do many NAS devices. Hashing protection and original floppies themselves are not additional copies. Hashing isn’t going to save data from a disaster (wind, power, tornado, fire, flood, physical destruction, theft, …). Original floppies do not count as a copy because this technology is dated, degrading, and getting harder to recover, a.k.a. legacy.
Recovery & file conversions
If you really, really, really want data back and are unable to recover it yourself, there are data recovery services. One ham said a person he knew used Gillware. A channel on YouTube I follow gave a recommendation to WeRecoverData. I did not use, research, or vet any of these companies so some due diligence is required before some phony-baloney service makes off with precious data. Gillware is a Micro-Center partner and WeRecoverData has a large number of companies that have used their services. Take that for what it’s worth.
Going back to my data that I “really, really, really” wanted to save from unreadable floppies. It turns out, I apparently told my younger self: self, you should make copies of these floppies onto other media or you may not have that data in the future. I copied all those important floppies, that are now unreadable, and burned them to CD. See, had two copies of data! I found those copies on a CD spindle of burned discs. Probably had read issues back then and saw the writing on the wall about floppies somewhere in the early 2000’s, near as I can tell. Writable CDs were reasonably priced about that time as well, $0.20, $0.30, $0.50, maybe as much as $1/each. I definitely didn’t know I could make an image of a floppy back then because the CD was drag-and-drop copies of the data. Better than not having it. Saved myself a lot of agony – although it doesn’t make as good of a story…
I didn’t touch on file conversions as the goal was to preserve data and I didn’t need to convert file formats. This may be needed in cases where proprietary programs were used and those companies no longer exist. The data can be read from the media but the file itself cannot be opened by any modern program. A copy of the original program used to create the file is best as there is likely some way to get that program running again. If it was an early version of a program that still exists, they may have changed data formats along the way and the earlier format is no longer readable by a modern version of the same program. Possibly filters or converters can be downloaded and installed.
Searching the Internet for the program originally used to create the file may lead to threads and worm-holes. Using an example of a very old word processing file, a similar-type program may be able to open the file such as Microsoft Office Word, Corel WordPerfect, LibreOffice Writer or legacy versions of old office suites like StarOffice or OpenOffice may improve chances. The more proprietary and obscure the program and format of the file, the harder it will be to find a program to read or convert the file, whereas open source programs and formats are likely to still be around 20 years later. I found enthusiasts will write free/open source programs to convert random obscure formats on GitHub.
Now that I’m done getting data off 3.5″ floppies, they’ll get destroyed for security reasons and donated to the circular file. I don’t see a reason to hang on to them seeing how many had read errors and now I have good copies of everything I want. I’m starting to see the same writing on the wall too with CD-R/DVD-R discs. A couple gave me read errors. Using another drive read the discs just fine. Still hanging on to those CD/DVD discs, until I get tired of looking at them.
While making disc images, I saw a name that sounded familiar: CMC Magnetics. Where have I heard that name before? If you were serious into your writable media mid-to-late 2000’s, a quick Internet search recalled memories of CMC being some of the cheapest & crappiest writable CD-R and DVD-R media available. Quality was not consistent, even between different spindles of the same brand. Verbatim was considered the gold standard for writable media. Even they got out of the market, selling off manufacturing to CMC. I think it’s the right time to get those important CDs and DVD-Rs imaged as well due to media quality concerns 😀
Epilogue:
I had one MacOS formatted floppy disk from grade school containing games that appears to use the HFS file system. Not much has gotten me anywhere near being able to play the games on that disk. I came across information for those needing to recover legacy Macintosh disks that’s worth passing along. Back in the early days, Apple developed a proprietary floppy disk technology to get more data on a standard 3.5″ floppy disk. Granted, everyone was trying to do their own proprietary formats to lock consumers into their technology and securing income for their company. Like proprietary technologies before and after it, users get shafted. The specific drives used to write those disks are the only ones that can read those floppy disks. Adding insult-to-injury, all recent MacOS versions have dropped support for those formats leaving users to find a branded drive with an appropriate legacy MacOS version should they want data off those disks. Not to mention, how will they get that data to a new system? A number of sources point to this website which has a lot of information to help MacOS users get their data recovered. Guess I’ll keep hoping to pick up an LC III or a LC 5xx from my early memories of MacOS.
Thanks for reading and 73… de Jeff – K8JTK
