Category Archives: software

Rescuing DVD-RAM Recordings from Obsolescence

Posted on by
Reply

I bought my first video camera more than 30 years ago. I went for Hi8, a higher resolution version of Video8, as opposed to VHS or VHS-C which was also popular at the time. Since then I have switched device and formats several times. There was always the worry that I would lose access to my recordings as devices able to read the old media become obsolete or die or the recording media themselves fail from old age. Losing irreplaceable videos of our kids when they were little is something I really didn’t want to experience. Here is a short summary of how I have been dealing with these challenges.

Hi8 (PAL) – early 1990s
I bough my first camcorder when I still lived in Germany so naturally it used the PAL standard (625 scan lines, 50 Hz). I did a lot of analog video editing using an S-VHS VCR, which could interface to the camcorder using S-Video cables. Even after I moved to Japan which uses the NTSC standard (525 scan lines, 60 Hz) I kept recording in PAL. A Samsung multi-standard recorder allowed me to record from the camcorder to NTSC VHS tapes. I also bought a multi-standard analog TV that could display PAL, SECAM and NTSC. However, for many years I just collected the Hi8 PAL master tapes in a cardboard box.
Along came Digital8, a successor to Hi8 that as the name indicates used digital recording but was backwards compatible with Hi8 and could play the old tapes. So eventually, as Hi8 camcorders were already becoming obsolete, I bought a second hand one off eBay when I was visiting Germany. It had an IEEE 1394 (Fireware) connector that made it possible to copy digital video to a computer equipped with that interface. I experimented with PCs with plug-in IEEE 1394 cards, but ultimately it was a Mac mini that allowed me to copy the old Hi8 PAL tapes to a hard disc using the German Digital8 camcorder, a Firewire cable and iMovie which was bundled with macOS. The output files were “.dv” files. Some tapes were difficult to load and took many tries before the camcorder would even play them, but I was largely successful.

Hi8 (NTSC) – late 1990s
When my kids started going to kindergarten I finally switched to a Japanese camcorder, still a Hi8 model but for the NTSC standard (US/Japan). Like with the PAL camcorder I saved all the tapes in a box. The Samsung multi-standard VCR developed issues and we bought a new S-VHS VCR equipped with a DVD drive. It supported DVD-R, DVD-RW and DVD-RAM, the latter with caddies (cartridges). It also supported S-Video. It was relatively easy to use the S-Video interface to copy from the Hi8 (NTSC) camcorder to double sided DVD-RAM media. About 2 hours of video would become one 4.2 GB file on DVD-RAM. I chose DVD RAM because it supposedly was more robust than DVD-RW (especially with the protective case), but as Blu-ray came along DVD-RAM became less and less common, with many DVD multi drives not supporting it any more. In 2008 and 2010 I made a stack of 6 double sided DVD RAM media that held video from 21 Hi8 NTSC tapes, but when the DVD section of the VCR died, I no longer had anywhere to play them.
This year I finally bought a USB 3.1 DVD drive that also supports DVD-RAM, though without support for caddies. I went for the BUFFALO DVSM-PTV8U3-BK/N (2180 yen, about US$21). It worked very well once I removed the DVD-RAM disks from their protective caddy. I hooked it up to a Windows 10 machine, plugging the two USB cables into different USB ports (USB 2.0 for power, USB 3.1 for data). I copied the entire folder structure on each side of the media to a separate new folder on the server hard disk. The actual video information on a DVD RAM disk is in a file called VR_MOVIE.VRO which is found inside a folder called DVD_RTAV. The open source VLC player will play this .vro file, as well as the .dv file captured on the Mac mini from the Hi8 recorder.

MPEG (NTSC) hard disc recorder – 2000s
After the various Hi8 recorders I moved from tape to a hard disk based camcorder, a Toshiba Gigashot GSC-R30. This used a Toshiba-made 1.8″ notebook hard disk. It also had a USB 2.0 interface and could be connected to any PC. The MPEG files would play on any MPEG player with support for its audio codec, including VLC. Therefore backing up and preserving these videos was pretty painless.

Smartphones – 2010s and beyond
The Toshiba GSC-R30 was the last camcorder I ever bought. Occasionally I still shot video on my Nikon D3300 DSLR camera, but mostly I moved on to mobile phones which may not have had an optical zoom or as much recording capacity, but they were always in your pocket and so easy to use and the quality improved with each generation.

Don’t lose your media!
If you value the images and videos you recorded over the years, make sure to migrate them to recording media that you can access for years to come and keep doing that. Also make sure you have backups. Tapes, DVDs, hard disks and SSDs will all become unreadable at some point. Don’t keep irreplaceable files on one laptop and hope that it will work forever because it won’t. At the very least, buy a USB drive and make a backup copy. Even better, buy another USB drive, make another backup copy and give it to someone else in your family. It’s better to have your valuable data saved in more than one place.
I am a great fan of the VideoLAN VLC media player. You can throw just about any video or audio format at it and it will be able to play it. I highly recommend it! 🙂

Rakuten Mobile and Google Pixel 3 / Pixel 3a

Posted on by
1

In mid-2019 the time came to change my mobile phone contract. For some reason mobile phone carriers in Japan do not want to reward your loyalty if you stick with them beyond the initial contract period and instead stiff you through higher prices or reduced discounts once the contract renews after 24 or 25 months. To me that’s just stupid, but anyway that’s how I had ended up with a Softbank Mobile contract and a new Google Pixel 3a after two years with UQ Mobile.

Now Rakuten Mobile had entered the ring to challenge the big three operators (docomo, au and Softbank), offering their “Rakuten UN-LIMIT V” which is free for 12 months for the first 3 million customers and has no penalty for cancelling after that. On paper it’s an offer with which you can’t loose. Rakuten’s network is still being built up from scratch, but the company has a roaming deal with au to give sufficient coverage until then. The plan includes 5 GB of data per month in the roaming area as well as 2 GB per month in 66 countries abroad (most of the destinations for international business travelers). I was willing to give it a try, even though they did not explicitly guarantee an unlocked Google Pixel 3a would work with their network. But we’re talking about Google, right? This is a trillion dollar company making some of the best phones outside of Apple and responsible for the operating system running on 3 out of 4 smartphones worldwide.

So I took the leap of faith and applied for MNP (mobile number portability) from my old carrier. Then I applied for “Rakuten UN-LIMIT V”. A few days later an envelope arrived from Rakuten Mobile with a new SIM card. When I activated the new SIM, the old Softbank SIM was disabled and the phone lost network access. I followed the printed instructions for swapping the SIM card but no joy: The new SIM card didn’t start working. Even after I manually selected the new APN (it wasn’t activated by default), the phone did not receive a phone number and there was no mobile data access, only WiFi. I googled the problem and found some blog posts with work-arounds that only work on Rakuten’s own network, not the roaming areas. Even so, I couldn’t make it work. My Pixel 3a didn’t even see any Rakuten network in the area where I live, densely populated Setagaya in Tokyo. I even tried the Rakuten SIM in my other phone, a Pixel 3 purchased off Amazon in the US. Same problem, no data. I called support and they told me what I already knew, that the Google Pixel 3 and Google Pixel 3a are not on their supported list. The list of supported devices for the Rakuten UN-LIMIT V service mentions the Google Pixel 4, Pixel 4 XL and Pixel 4a but no Pixel 3 or 3a. For Rakuten with Docomo lines or au lines they list the Docomo Pixel 3a as the only compatible device from Google.

Rakuten Mobile support told me I could either buy a different phone or transfer the number to another provider or give up the number altogether.

So I requested an MNP number from Rakuten Mobile (3000 yen + tax) to move on and applied for a voice + data SIM from a Mobile Virtual Network Operator (MVNO). Three days later I had that SIM in my mailbox. I plugged it in, followed the phone-based activation procedure and 10 minutes later had a working phone again, with voice, SMS and 3 GB of data for 1600 yen a month + tax.

I really hope Rakuten will get its act together. The Japanese oligopoly of docomo, au and Softbank with difficult to understand but ultimately overpriced plans needs some real competition. But if they are serious, how can they not support Google’s flagship phones? Rakuten blew a one time chance to gain a loyal customer. Meanwhile I’m happy with IIJmio, the service I signed up for.

Archiving/Exporting Skype Chats

Posted on by
Reply

Many long time Skype users have been unhappy how the service has changed over the years, especially after the company was acquired by Microsoft. I am one of them.

Starting out as a fairly powerful, secure and private peer-to-peer network, Skype gradually mutated into a client-server product with limited privacy and significantly reduced functionality and performance.

Every now and then I used to make backups of important Skype chats, so I would have the information available for future reference. This used to be doable with copy and paste, but that no longer works. Skype then offered a function to export the chats to a file, for viewing with a viewer.

In late 2017 that chat export functionality was removed from the macOS client in the upgrade from version 7 to version 8. Everything typed in chats after that upgrade no longer gets tracked on the client side but ends up on the server, where of course it is no longer private. According to Microsoft, the chat archives going back as far as April 2017 will be maintained until you delete them.

Skype chat archives after April 2017

To download chat archives, log into your Skype account from a browser using this link:

https://go.skype.com/export

Select what you want to download and click “Submit request”. You will be notified once an archive of the chats is ready for download. Alternatively, you can go back to above export link to check manually. Once you download and extract the archive, there’s a Javascript viewer for it.

Skype chat archive until 2017

For older chats, the macOS Skype client has an option to export the locally saved chats. Go to:

Setting / Messaging / Export chat history from Skype 7.x

Pixel 3a: “Analogue audio accessory detected. The attached device is not compatible with this phone”

Posted on by
1

Three months ago my wife and I both changed our local smartphone plans and changed to a Google Pixel 3a.

Within a week she had a problem where her phone would suddenly shut down when the battery still had 50-60% charge left, while doing nothing. This even happened after the phone had been factory reset, with no third party apps installed. It took a few weeks before Softbank Mobile replaced the phone under warranty.

Now it’s my turn it seems. Recently I would find the phone with less than 50% of battery left in the morning when I had left it hooked up to the charger (so it should have been at 100%). I would reconnect the phone but it would not show the charging symbol. I then tried different cables and chargers and also rebooted the phone. Eventually it would charge again.

Today it showed the following error:

Analogue audio accessory detected. The attached device is not compatible with this phone

Nothing was plugged into its headphone socket or into the USB C port. There was no analog audio device connected. I googled the problem and most results suggested it was a problem with the USB C port. That makes sense, since it would explain both the charging issues and the audio warning, as one can plug an analog device into the USB C port via an adapter.

Other results suggested a factory reset may get it working again, but that did not work for me. After the factory reset I could charge the phone with the charger from my earlier Nexus 6P and a USB C cable when powered off. However, after I powered it up and started the system restore, it no longer charged from either that charger or from the Pixel 3a charger or a Pixel 3 charger. The problem was back.

Time to take it back to the Softbank Mobile shop, I guess 🙁

Needless to say, I am not impressed with a 2/2 failure rate for our Pixel 3a phones so far. The Pixel 3a was great while it worked. The picture quality seems as good as for its more powerful sibling, the more expensive Pixel 3 and battery life was decent too. But that is all meaningless if it randomly shuts down or you can no longer charge it.

UPDATE: Softbank Mobile sent the two months old Pixel 3a in for testing and repair. A couple of days later they quoted almost 20,000 yen (US$190) for the repair of the unspecified damage, which they said would not be covered under warranty 🙁

Outlook Express Error 0x800CCC0B and the End of TLS 1.0 (Deprecated SSL Protocol)

Posted on by
8

Microsoft Outlook Express (OE) is an obsolete mail client that was available in Microsoft Windows XP, Windows 2003 Server and older Microsoft operating systems. It was no longer available on Windows Vista and later, though Windows Live Mail is relatively close in user interface and appearance.

Despite being obsolete and only working on operating systems no longer supported or updated by Microsoft, it still has some users who prefer its simple but powerful user interface. Some of those users will have had a frustrating experience recently, when various mail servers stopped working for outbound mail in OE. Specifically, these are mail servers that use SSL on submission port 465 or 587 for SMTP.

Secure Socket Layer (SSL) is a mechanism for encrypting data between a client and a server. You may know it from website URIs starting with “https:” and web sessions displaying a padlock symbol next to the URI. There are various protocol versions that can implement this encryption layer. One of these, TLS 1.0 which was conceived in 1999, has now been officially deprecated (made officially obsolete) as of the end of June 2018. Software now has to use more recent protocols, such as TLS 1.1, TLS 1.2 or the recently defined TLS 1.3.

Unfortunately, TLS 1.0 is all that OE will speak. It does not understand TLS 1.1 or later. Therefore it can not pick up mail from a POP server using SSL on port 995 or an IMAP server on port 993 or send mail to an SMTP server on port 465 (or 587) with SSL enabled.

Workaround
The only workaround I am aware of (other than switching to a more modern mail client) is to use Stunnel, a tool for Windows or Linux that acts as a proxy. You can configure it to establish an SSL connection to a given host and port when a connection to a given local port is made. Thus you could configure OE to connect to port 9465 on the machine running Stunnel, which might then connect via SSL to smtp.example.com:465 using a more modern TLS version supported by Stunnel (but not directly by OE).

Example
Let’s say Outlook Express was configured to submit outbound mail to smtp.outboundmailserver.com, port 587 via SSL/TLS. This is our SMTP server. Once this server refuses to allow TLS 1.0 connections, Outlook Express will no longer work. Let’s say we also have a simple Linux server mylinuxserver.com. This could even be something like a Raspberry Pi single board computer booting off flash memory. It can run on a local IP in our LAN, if you don’t need to have access from outside your building (OE running on a desktop). On this server we install the stunnel package:

sudo yum install stunnel

Please read the documentation on how to enable the service and have it auto-start when the Linux server reboots.

Next we configure stunnel to act as a client on our behalf and configure it to accept TLS 1.0 connections from us and forward them to the real POP3, SMTP or IMAP server using the latest TLS on our behalf. We will create lines like these in /etc/stunnel/stunnel.conf:

client = yes

;cert = /etc/pki/tls/certs/stunnel.pem
;sslVersion = TLSv1
;chroot = /var/run/stunnel
;setuid = nobody
;setgid = nobody
;pid = /stunnel.pid
;socket = l:TCP_NODELAY=1
;socket = r:TCP_NODELAY=1

[smtp-outboundmailserver]
accept = 1587
connect = smtp.outboundmailserver.com:587

Create other entries for the services that you need TLS support for and restart the stunnel service. Then reconfigure Outlook Express to access the Linux host and the port number listed with “accept = ” in place of the original server that refused your Outlook Express TLS 1.0 connection. You should be good to go!

Long term you will still need to migrate to another mail client such as Thunderbird, Windows Mail or OE Classic, but this workaround will buy you some time for that.

Picasa: “Failed to download album list”

Posted on by
33

If you are still using the Picasa 3 desktop application by Google and got the above error message, here’s some bad news for you: Google has finally killed this app. On March 26, 2018 they announced that it would no longer be able to upload new albums. So this error message is not temporary and there is no direct fix.

I think it’s very regrettable that Google has been killing off Picasa step-by-step. This is only the latest nail in the coffin. I had been using Picasaweb and Picasa since 2010 and they were great products.

The good news is that you can still create albums from folders using a web browser. Say you have a folder named “2018-03-26 Cherry Blossom Party”. Just follow these steps (for Windows and Chrome):

1) Select its parent folder in Windows Explorer, then slowly click on the folder that you want to upload, twice: Once to select it, then once more to enable you to edit the folder name as if to rename it. When the name becomes editable, press Ctrl+C to copy the folder name, then press Esc to keep the name unchanged. This stores the folder name in the copy-and-paste clipboard, which will save you from having to manually retype the name later.

2) In Chrome, go to https://photos.google.com/ and click on “Upload” (on the top right). A file selector dialog will open up. Click through to the contents of the folder you want to upload. Select all files in the folder using Ctrl+A and click “Open” to confirm the upload.

3) The browser will upload all files and give you a choice of “Add to album” or “Shared album”. Select “Add to album”. To create a new album with the name of the folder, select “New album”. Click on the album name showing as “Untitled” and use Ctrl+V to paste the name you copied in step 1. Hit Enter and click on the check mark to confirm creation of the new album.

Voila, you have a new album online, with the same name as the local folder. Repeat as needed for multiple folders. This is as simple as it gets without the old Picasa app.

Strava Cycling Climbing Challenges

Posted on by
1

Strava is a popular service for logging bike rides and other activities, which provides a way of comparing one’s achievements with those of other cyclists and runners. Competition is a powerful stimulant and a main driver behind the success of the service. Monthly “challenges”, such as a Gran Fondo (a ride of at least 100 or 130 km, depending on the time of the year) or a monthly cumulative distance or elevation gain challenge, are particularly popular on Strava.

While I regularly participate in the Cycling Distance and Gran Fondo Challenge, I do not normally sign up for the Cycling Climbing Challenge, which is meant to encourage you do ride hilly courses. I love hilly courses. In fact, most of my weekend rides are hilly, usually going from close to sea level to over 900 m and back.

Last year I averaged one century ride (at least 160.9 km / 100 miles) about every other week, so the Gran Fondo challenge is not really that much of a challenge for me. My typical centuries are about 170-190 km with 1800-2100 m of elevation gain. Yet at 7500 to 8000 m the goal for the Cycling Climbing Challenge is set so high, I could do a hilly century ride four Saturdays in a row and still miss the climbing goal. So how do other people, who do not ride 170 km into the hills every other weekend complete the climbing challenge?

I think the Strava climbing goals are designed for people who record their rides with phones or other GPS devices that rely only on satellite data for elevation. GPS-based elevation data is much less precise than lattitude-longitude data. Other popular GPS units like the Garmin Edge 1000, Garmin Edge 520 (or my o_synce Navi2coach) use a barometer for more precise elevation tracking. The problem with GPS-based elevation is that it’s noisy, it will go up and down pretty randomly but all those little ups will be added up by Strava, resulting in a considerably inflated climbing total. If you’re using a GPS device measuring relatively accurate barometric elevation, you can’t really compete against all that noisy data 🙁

I could confirm this in group rides with other people who were using mix of equipment, where I had a chance to compare the posted stats on Strava afterwards. The iPhone or Android-app recorded totals were often 50-100% higher than the Garmin-recorded totals, for one and the same course.

Here is one random example of 4 people doing the same course up a volcano in Tenerife, Gran Canria yesterday. Note, this not my ride, I just randomly stumbled on it while looking at high scorers in the March Cycling Climbing Challenge on the Strava website. Two of these cyclists were using the Strava iPhone app, the other two were using a Garmin Edge 520:

Strava iPhone App:
https://www.strava.com/activities/1437744625
5,080m

Strava iPhone App:
https://www.strava.com/activities/1437728113
4,635m

Garmin Edge 520:
https://www.strava.com/activities/1437709764
2,847m

Garmin Edge 520:
https://www.strava.com/activities/1437730812
2,556m

As you can see, the two cyclists using the phone app posted almost double the total climbing for the course as the Garmin users, despite riding the very same roads and posting the same elevation profile for the activity (i.e. no hill repeats).

Based on evidence like that, I don’t think elevation gain competitions on Strava are happening on a level playing field! 😉

Huawei Nexus 6P Battery Upgrade

Posted on by
12

I’ve had my Huawei Nexus 6P for about two years now. The combination of a great camera, an excellent screen, good performance and decent battery life has made this my best smartphone ever.

However, a couple of months ago something happened as the battery capacity appeared to have collapsed dramatically. Sometimes the phone would shut down only 5 hours after I had disconnected it from the AC charger when I left home, starting off supposedly fully charged! I had to always carry a USB battery and cable with me to not risk losing the use of my phone in the middle of the day.

Attempts to recalibrate the capacity indicator helped only insofar as the phone would shut down at 14% charge instead of say 55% charge, so there was slightly more warning, but the number of hours was still too short. This actually seems to be a common problem with the Nexus 6P, which otherwise is still a great phone.

It’s not uncommon for Li-ion batteries to significantly lose capacity after about about three years, but if it happens after less than two years as in my case, that’s not very good. Fortunately, replacement batteries are available and any competent phone repair shop will be happy to do the necessary surgery to replace a battery that is on its way out. Unfortunately the days when you could simply pop open the phone case without any tools and swap the battery yourself are long gone. This is a trend started by Apple and almost every other phone maker has since followed suit. I think it’s meant to get people to buy a new phone sooner, which is good for Apple and its competitors, but bad for consumers and for the planet.

There are Youtube videos that will show you how you how to open the Nexus 6P case and disassemble the phone to swap the battery. This involves the use of a hairdryer or heat gun to soften the glue that holds it all together as well as a plastic card and a small screw driver. As I did not feel adventurous enough to attempt this myself, I contacted several phone repair shops here in Tokyo. Repair King Japan replied. Though they they didn’t have the Nexus 6P battery in stock they were happy to order one for me. Once they got it, I dropped the phone off and two hours later I could have it back with a new battery. So far it’s looking good: It’s been 40 hours since the last full charge (with battery saver mode inactive) and it’s still showing 64% with about 3 days of power left 🙂

UPDATE: At 72 hours, it still had 23% charge left. At that point I connected it to a charger.

Hopefully with the new battery my Nexus 6P will be a great phone again for a few more years!

Updated jwhois.conf File for CentOS for New gTLDs

Posted on by
1

The whois command on CentOS 6.x and 7.x doesn’t handle queries for many domains in new Top Level Domains (TLDs) that were added by ICANN in the last few years.

Domains from many of these new TLDs are selling as cheap as $0.99 a pop, making them attractive to snowshoe spammers who create them in large numbers. As a spam researcher, I see lots of new spam domains from TLDs such as .xyz, .online, .top. .club, .services, .win, .site, .bid, .life and .trade.

WHOIS is an important tool for me to track the domain registrants. CentOS uses jwhois as its WHOIS client, which relies on a configuration file to tell it what servers to query for detailed information. The configuration file that comes with recent CentOS versions is woefully out of date.

I have gone through the currently existing TLDs and counted 466 of them that are not supported by jwhois but appear to have a valid WHOIS server. I have been able to verify for about half of these TLDs that the WHOIS server works and have added them to my configuraion file, which you can download here.

Many of the rest of the new TLDs are hosted on Neustar, which performs rate limiting on lookups. Because of that I didn’t fully verify functioning of all those hosts, but I verified that CNAMEs exist for the WHOIS hosts that redirect to Neustar WHOIS servers and tested a small sample of those TLDs.

Getting Rid of the EMUI Launcher on the Huawei P9 Lite

Posted on by
Reply

Last time I switched mobile provider here in Japan, I signed up for a contract that included a Huawei P9 Lite. My biggest grip about it is its non-standard EMUI interface that runs on top of Android 6.0.1.

Previously I was using a Nexus 5, which had worked OK for me, though the picture quality of its camera was rather mediocre. One nice thing about the Nexus 5 was that it runs stock Android, with no customization. Its user interface is identical to that of my other phone, a Nexus 6P.

I really prefer stock Android without OEM customization. For one, stock Android means you can get version upgrades sooner and for longer (or at all!).

I found the EMUI launcher confusing. For example, I did not see any easy way to launch an app that didn’t have a desktop link.

It’s possible to switch from EMUI to the standard Google launcher. Here are the steps I performed:

1) Install “Google Now Launcher” via Play Store.

2) Swipe down, select Shortcuts and then Settings

3) Enter “def” into the search box at the top (may have to scroll up first)

4) Select “Default app settings”

5) Select “Launcher” and pick “Google” instead of “Huawei Home”. Ignore the warning that tries to scare you into sticking with EMUI (you can always change back by following the same steps and selecting “Huawei Home” again).

6) There you go!

The irritating long push home button

Another irritation that seemed to happen more on the Huawei than on my other phones was the Google screen that pops up (seemingly randomly) when I just try to go to the home screen. It has a “Want answers before you ask?” prompt at the bottom and a Google search box with voice search option at the top. I really don’t need this screen because the standard Android home screen already has a Google search bar at the top. I’d rather have the home screen with all my app shortcuts come up reliably whenever I push the Home button!

It took me a while to figure out that this Google search screen comes up on what the phone thinks is a long push of the Home button, which has a different meaning from a regular short tap. If that happens, just tap again and it will go to the home screen. Or just make it your habit to double tap the home screen to go to the home screen, then this should never happen 🙂