TIL I learned how (and how NOT) to restore my Mac backup

They say you never know if you have a real backup until you try to use it. They’re right.


  • Don’t encrypt your drive until you’ve restored your data
  • Don’t exclude things from Time Machine backup

The details

I sent my 2017 MacBook Pro to Apple because the keyboard was acting up. And as long as I was at the Genius Bar, I thought it’d be a good time to mention a weird problem I’d had when using Screen Mirroring on my Apple TV (moving the cursor would reveal things lower in z-order – it was weird); the Genius duly wrote that down and sent my machine to the depot for repair.

A few days later, the machine was back in my hands; not only had they replaced the keyboard and the top case (as I’d expected), but they also had replaced a few other parts – and they’d either wiped or replaced the SSD, so I had a virgin installation of Mac OS and none of my data. I booted up the machine, and, for security, told it to turn on FileVault to encrypt the SSD; then I set about restoring my data.

Plan A

I had multiple backups: Time Machine, a bootable clone drive (made with SuperDuper!), and Backblaze for offsite backup. I thought that the easiest solution would be to reboot from the clone drive and restore to the SSD, so I did exactly that.

The restore went fine, except for one small problem – SuperDuper! was unable to update the PreBoot environment. When I tried to boot from the SSD, I got a blinking folder with a question mark, which meant that the system couldn’t find the startup disk. I booted from the clone again and used System Preferences to set the startup disk to the SSD; this time, instead of the question mark, I was asked for a disk password…which, of course, I didn’t have.

Back to the clone, where I could easily mount the SSD and see its contents. Searching gave me this thread, but I couldn’t figure out how to apply the hints to my environment, so it was time for Plan B.

Plan B

I booted the “Install High Sierra” USB stick I’d kept around from last year and erased the SSD. Then I restored from my Time Machine backup. Many hours later, I was able to boot the SSD, and all seemed well.

Except…my desktop background was wrong. And my Dropbox folder was empty (though that was easy to fix by firing up Dropbox). And when I looked further, I realized that I didn’t have any of my photos, nor my iTunes library, nor my DevonThink databases. It was as if they’d never existed. But they were on the clone drive. Hmmm….

Plan C

I booted the clone and fired up SuperDuper! again. And, again, I restored the SSD from the clone, using “Smart Copy”, which found 150GB already on the SSD, and about 200GB of data that needed to be copied.

This time, when I booted the SSD, all was well – I had all of my data.

I opened Time Machine Preferences and discovered that I’d excluded my iTunes library, my Pictures folder, and my DevonThink databases from being backed up in order to save space on the drive in the Time Capsule that I was sharing among several computers. Oops. Especially since I’d stopped sharing a Time Capsule months ago and was backing up to a directly-attached 4TB drive, with plenty of space.


All seems to be well; I’ve removed the exclusions from Time Machine and it’s busily backing up everything now.

I haven’t checked to see if the Apple TV screen mirroring problem has been fixed – I guess I should find out, though.

I think I’ll wait a few days after Mojave ships before I upgrade; I really don’t want to go through this again any time soon!

Posted in Computer Stuff | Comments Off on TIL I learned how (and how NOT) to restore my Mac backup

TIL how to build Python 3.7 with statically-linked libssl and libcrypto

I use a Virtual Private Server on DreamHost to run the Toastmasters District 101 website. For the most part, I’m happy with their service, and with a shell prompt, it’s usually easy to install whatever software I need.

Except Python 3.7. Python 3.7 requires a newer level of OpenSSL than DreamHost offers, and since I don’t have root access on a VPS, I can’t just replace OpenSSL. Compiling a current version and installing it in a directory ($HOME/usr/local) was easy enough:

./config --prefix=$HOME/usr/local --openssldir=$HOME/usr/local/openssl   
make test   
make install   

Building Python 3.7 was also easy, but getting it to use my copy of OpenSSL was not.

At first, I tried adding my OpenSSL to LD_LIBRARY_PATH, which worked, but it made git complain: no version information available (required by ssh), and that seemed unfortunate (and made me worry that I might break other things).

After much searching, I found Python issue 21541, which had the hints I needed to statically-link my copy of OpenSSL into the Python executable.

First, run configure:

./configure --prefix=$HOME/opt/python-3.7.0 --with-openssl=$HOME/usr/local/

Then uncomment and change the section of Modules/Setup dealing with SSL to this:

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
_ssl _ssl.c \
    DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
    L$(SSL)/lib -Wl,-Bsymbolic $(SSL)/lib/libssl.a $(SSL)/lib/libcrypto.a

After that, the usual make && make install worked.

Posted in Life | Comments Off on TIL how to build Python 3.7 with statically-linked libssl and libcrypto

I’ve moved!

Ducklings on the move with mama duckI’ve moved this blog to Linode. It should run a little faster, cost me a little less, and let me do more things with the server.

I took the photo near my house a couple of days ago while I was out for a walk; I was surprised to see a mama duck and her flock of ducklings going the other way!

Posted in Life | Comments Off on I’ve moved!

In Praise of Venice Classic Radio

We like to have light classical music in the background while we’re at home. During most of the year, we usually listen to our local classical station, KDFC, or one of SiriusXM‘s classical channels. But once Thanksgiving rolls around, things change. “Holiday” music begins to creep into the playlist, and as the month goes on, it takes up more and more airtime.

But I’ve found an alternative – Venice Classic Radio. Their playlist is blessedly free from “holiday” music for most of the month – they do play the occasional real classical Christmas piece, but it’s a small part of their programming, and they mostly save it until Christmas Eve and Christmas Day. I heard Silent Night yesterday and Nutcracker today, both of which were very pleasant indeed; they also played Kol Nidre today, much to my surprise.

Check them out any time you want classical music without interruption (they play a 30-second multi-lingual station ID at the top of the hour, but that’s the only announcement you’ll hear) – and especially during December!

Happy Holidays to all, and to all a good night!

Posted in Life | Comments Off on In Praise of Venice Classic Radio

TIL why you “lay down” Port

When Jeff was four or five, I thought it would be a nice idea to buy a bottle of Port from his birth year and share it with him on his 21st birthday. I found a bottle of “Late Bottled” Vintage Port at Trader Joe’s (this was before we started going to wineries!) and put it aside.

Jeff’s 21st birthday came – but he was away at college. The bottle sat.

Years passed.

Jeff’s home for the holidays, and I thought New Year’s Eve would be a good time to finally share the Port. I took the bottle out of the closet and tried to open it.

The cork had disintegrated. I managed to get enough of the cork out of the bottle to pour the Port through a strainer and into our glasses – but it was undrinkable.

If only I’d read the back side of the bottle and followed their advice.

back label

Next time for sure!

Posted in Life | 1 Comment

TIL not to plug an Amazon Echo Dot into my Mac Mini

I recently put an Amazon Echo Dot in my home office. It needs USB power; rather than plug in yet-another-power-adapter, I plugged it into a spare port on a USB hub connected to my Mac Mini.

All was well…until I needed to reboot the Mac. Then I noticed two things:

  • A normal reboot took a long time to show the login screen (more than a minute, versus the usual 10 seconds)
  • Pressing Option during the reboot (to try to reboot from another device) failed – the machine hung, never giving me the screen with devices to try.

I tried resetting NVRAM and the SMC; no dice.

Eventually, I managed to reboot and took a look at the system log; there were many, many error messages like this:

Oct  6 12:52:00 office kernel[0]: 000573.331885 AppleUSB20HubPort@14233000: AppleUSBHostPort::disconnect: persistent enumeration failures

I started disconnecting devices, and when I unplugged the Dot, the messages stopped appearing.

I’m now powering the Dot with its own power supply, plugged into a power strip. It works great, and the Mac Mini is happy, too.

Posted in Life | 1 Comment

TIL about Python’s “enumerate”

I’ve been writing Python for more than 10 years, and I can’t count the number of times I’ve written code like this:

index = 0
for thing in array:
    do_something_with(index, thing) # Because I care about the index AND the item
    index += 1

But today I learned about the built-in enumerate function which does exactly the same thing, but avoids any chance of getting “index” and “thing” out of sync (not that I’ve ever made that mistake, of course…).

I’m not going to rewrite any old code for fear of adding new bugs, but I’ll remember to enumerate going forward.

Thanks, Dr. Drang!

Posted in Life | Comments Off on TIL about Python’s “enumerate”

Adding email addresses to Divi’s “person” module for District 101

The Divi theme is powerful, but it has some quirks. One of those is in the “person” module, which doesn’t provide a place for contact information (phone and email). I wanted to add email for the people listed on the District Leadership Team page; the obvious way was to add the email address to the text for each person, but that felt inelegant.

A quick Google search led me to a free plugin, Person Module Extended by Dani Dwiputra, made available through Divi Space. I installed the plugin, added emails in the appropriate fields, and could have called it a day, except that I wanted to make some changes to the presentation.

Here are the diffs to get the results I wanted:

++ Desktop/person-full-social/dd-person-modules.php 2016-07-24 21:12:21.000000000 -0700
@@ -465,2 +465,12 @@

+        if ( '' !== $member_email ) {
+            $contact = sprintf('<a href="mailto:%1$s" class="et_pb_member_email">%1$s</a>', esc_html( $member_email ) );
+            $contact .= ( '' !== $member_phone ? sprintf(' | <a href="tel://%1$s" class="et_pb_member_phone">%1$s</a>', esc_html( $member_phone ) ) : '' );
+        } else {
+            $contact = ( '' !== $member_phone ? sprintf('<a href="tel://%1$s" class="et_pb_member_phone">%1$s</a>', esc_html( $member_phone ) ) : '' );
+        }
+        if ( '' !== $contact ) {
+            $contact = '<p>' . $contact . '</p>';
+        }
        $output = sprintf(
@@ -471,4 +481,4 @@
+                   %7$s
-                   <p>%7$s  |  %6$s</p>
@@ -483,3 +493,3 @@
            ( '' !== $member_phone ? sprintf( '<a href="tel://%1$s" class="et_pb_member_phone">   %1$s</a>', esc_html( $member_phone ) ) : '' ),
-           ( '' !== $member_email ? sprintf( '<a href="mailto:%1$s" class="et_pb_member_email">   %1$s</a>', esc_html( $member_email ) ) : '' ),
+           $contact,
            ( '' !== $position ? sprintf( '<p class="et_pb_member_position">%1$s</p>', esc_html( $position ) ) : '' ),
@@ -493,2 +503,2 @@
-new DD_Builder_Module_Team_Member;
\ No newline at end of file
+new DD_Builder_Module_Team_Member;
diff -r -U1 Downloads/person-full-social/module-extend.php Desktop/person-full-social/module-extend.php
--- Downloads/person-full-social/module-extend.php  2016-07-20 10:58:09.000000000 -0700
+++ Desktop/person-full-social/module-extend.php    2016-07-24 21:15:17.000000000 -0700
@@ -9,2 +9,3 @@
  * License: GPL2
+ * Modified by David Singer

Thanks, Dani!

Posted in Life | Comments Off on Adding email addresses to Divi’s “person” module for District 101

Speeding up “Feed Them Social” for District 101

The District 101 Toastmasters website uses the Feed Them Social plugin to add our Facebook and Twitter feeds to our homepage.

It works well, but it can be slow – if it needed to go to Facebook to update its data, it could take as much as 10 seconds to build the page; even if data was in the plugin’s cache, it took a couple of seconds to build the HTML. You can see the shortcodes I had on the home page below.

I thought about writing my own program to mine the Facebook Graph and the Twitter Feed. It would let me produce exactly what I needed, but it would be a perpetual maintenance headache (and I don’t plan to be Webmaster forever). I needed a better answer.

And then it struck me – if I could figure out a way to use the plugin as part of a batch process and save the HTML, I could get the plugin’s processing time out of the critical path, and I wouldn’t have to keep up with changes to the Facebook and Twitter APIs. Here’s what I did:

  • Created a special version of the home page, with the calls to the plugin in the proper place in the page layout (just in case it mattered), but with none of the other features of the home page.
  • Surrounded the calls to the plugin on the special page with flag lines so I could easily pull out the HTML generated by the plugin, using a simple Python program.
  • Made the special page password-protected in WordPress to keep it out of the visible menu structure shown to site visitors.
  • Wrote a script to be called periodically by cron that:
    • used curl to fetch the special page
    • ran the Python program to extract the HTML from Feed Them Social
    • saved the result as a file
  • Replaced the calls to the plugin on the homepage with a couple of lines of PHP to include the saved file as part of the homepage

The result: the homepage loads at least 3.5 seconds faster and looks the same.

Here’s the relevant part of the special page (the calls to FTS are the same as those I used to have on the homepage).

Start Flag
[fts_facebook id=d101tm posts_displayed=page_only type=page]
[fts_twitter twitter_name=d101tm]
End Flag

Here’s the shell script:

# Update the cached static file for social media from Feed Them Social
secret=password for the WordPress page
cd ~/files/social
# The next operation will take a while, so we write to a temporary file
curl -sL "http://d101tm.org/wp-login.php?action=postpass" -d "post_password=$secret" -e "http://d101tm.org/path to the special page/" -b /dev/null | $mydir/updatefts.py > $outfile
# If all went well, we can replace the real file
if [ -n $outfile ]; then
    mv $outfile fts.html
    echo $outfile is zero length!
    exit 1

And here’s the Python program:

#!/usr/bin/env python
""" Extract the HTML generated by Feed Them Social from stdin, write to stdout"""

import sys
startflag = 'Something unlikely to appear on the page'
endflag = 'Something else unlikely to appear on the page'
havestart = False
haveend = False

def findendflag(s):
    """ Returns a tuple: (any data before the endflag,
                          whether the endflag was found) """
    if endflag in s:
        return (s.split(endflag,1)[0], True)
        return (s, False)

for l in sys.stdin.readlines():
    if not havestart:
        res = l.split(startflag,1)
        if len(res) > 1:
            havestart = True
            (l, haveend) = findendflag(res[1])
    elif not haveend:
        (l, haveend) = findendflag(l)
Posted in Life | Comments Off on Speeding up “Feed Them Social” for District 101

Some Thoughts on the Toastmasters District 101 Website

I’m Webmaster for Toastmasters District 101, which started operations on July 1, 2016. For the previous two years, I had been on the Webmaster team for Toastmasters District 4; District 101 covers the area from Mountain View to Monterey, which was in District 4 until July 1.

As Webmaster for District 101, I had to choose a CMS (I chose WordPress and a theme (Divi)). I had help from other members of the District 101 Foundations team in picking other plug-ins and in creating the content for the website.

This will be the first, and possibly not the last, in a series of postings about the website and the other code I write in support of it. I’m posting here for two reasons:

  • Perhaps it’ll help someone else
  • Perhaps I’ll be able to find it in the future!

You can decide which is the more important reason.

Tweaking The Events Calendar

We chose to use The Events Calendar from Modern Tribe as our calendar tool.

Out of the box, it works well, but I wanted to display some information about the kinds of events we would show on the District Calendar and how to search for an event.

The obvious way to do that was to use the “Add HTML before event content” feature in Advanced Template Settings; unfortunately, when I added the information there, it not only appeared on calendar listings, it appeared when you chose a single event to display (and in that case, there was no search bar available, so it was really confusing!).

I found two ways to get the kind of display I wanted:

  • Override default-template.php by copying it from the-events-calendar/src/views/ to my-child-theme/tribe-events/ and then only calling tribe_events_before_html if I wasn’t displaying a single event; this has the advantage of making the information editable in the Events Calendar settings.
  • Add a new action for the tribe_events_bar_before_template hook that would display my information if and only if the search bar was going to be displayed. This has the disadvantage of requiring me to hard-code the information in my action.

I decided to go with the first choice, which also required me to be sure to use the “Default Events Template” instead of the “Default Page Template” in the Events Calendar settings.

Here’s the diff to create my modified default-template.php file:

@@ -17,7 +17,11 @@
 <div id="tribe-events-pg-template">
-        <?php tribe_events_before_html(); ?>
+    <?php
+        if (!(tribe_is_event() && is_single())) {
+            tribe_events_before_html();
+        };
+    ?>
         <?php tribe_get_view(); ?>
         <?php tribe_events_after_html(); ?>
 </div> <!-- #tribe-events-pg-template -->

Posted in Life | Comments Off on Some Thoughts on the Toastmasters District 101 Website