Speeding up “Feed Them Social” for District 101

The District 101 Toastmasters website uses the Feed Them Social plugin to add our Facebook and Twitter feeds to our homepage.

It works well, but it can be slow – if it needed to go to Facebook to update its data, it could take as much as 10 seconds to build the page; even if data was in the plugin’s cache, it took a couple of seconds to build the HTML. You can see the shortcodes I had on the home page below.

I thought about writing my own program to mine the Facebook Graph and the Twitter Feed. It would let me produce exactly what I needed, but it would be a perpetual maintenance headache (and I don’t plan to be Webmaster forever). I needed a better answer.

And then it struck me – if I could figure out a way to use the plugin as part of a batch process and save the HTML, I could get the plugin’s processing time out of the critical path, and I wouldn’t have to keep up with changes to the Facebook and Twitter APIs. Here’s what I did:

  • Created a special version of the home page, with the calls to the plugin in the proper place in the page layout (just in case it mattered), but with none of the other features of the home page.
  • Surrounded the calls to the plugin on the special page with flag lines so I could easily pull out the HTML generated by the plugin, using a simple Python program.
  • Made the special page password-protected in WordPress to keep it out of the visible menu structure shown to site visitors.
  • Wrote a script to be called periodically by cron that:
    • used curl to fetch the special page
    • ran the Python program to extract the HTML from Feed Them Social
    • saved the result as a file
  • Replaced the calls to the plugin on the homepage with a couple of lines of PHP to include the saved file as part of the homepage

The result: the homepage loads at least 3.5 seconds faster and looks the same.

Here’s the relevant part of the special page (the calls to FTS are the same as those I used to have on the homepage).

Start Flag
[fts_facebook id=d101tm posts_displayed=page_only type=page]
[fts_twitter twitter_name=d101tm]
End Flag

Here’s the shell script:

# Update the cached static file for social media from Feed Them Social
mydir=$PWD
secret=password for the WordPress page
cd ~/files/social
# The next operation will take a while, so we write to a temporary file
outfile=newfts$$.html
curl -sL "http://d101tm.org/wp-login.php?action=postpass" -d "post_password=$secret" -e "http://d101tm.org/path to the special page/" -b /dev/null | $mydir/updatefts.py > $outfile
# If all went well, we can replace the real file
if [ -n $outfile ]; then
    mv $outfile fts.html
else
    echo $outfile is zero length!
    exit 1
fi

And here’s the Python program:

#!/usr/bin/env python
""" Extract the HTML generated by Feed Them Social from stdin, write to stdout"""

import sys
startflag = 'Something unlikely to appear on the page'
endflag = 'Something else unlikely to appear on the page'
havestart = False
haveend = False

def findendflag(s):
    """ Returns a tuple: (any data before the endflag,
                          whether the endflag was found) """
    if endflag in s:
        return (s.split(endflag,1)[0], True)
    else:
        return (s, False)

for l in sys.stdin.readlines():
    if not havestart:
        res = l.split(startflag,1)
        if len(res) > 1:
            havestart = True
            (l, haveend) = findendflag(res[1])
            sys.stdout.write(l)
    elif not haveend:
        (l, haveend) = findendflag(l)
        sys.stdout.write(l)
Posted in Life | Comments Off on Speeding up “Feed Them Social” for District 101

Some Thoughts on the Toastmasters District 101 Website

I’m Webmaster for Toastmasters District 101, which started operations on July 1, 2016. For the previous two years, I had been on the Webmaster team for Toastmasters District 4; District 101 covers the area from Mountain View to Monterey, which was in District 4 until July 1.

As Webmaster for District 101, I had to choose a CMS (I chose WordPress and a theme (Divi)). I had help from other members of the District 101 Foundations team in picking other plug-ins and in creating the content for the website.

This will be the first, and possibly not the last, in a series of postings about the website and the other code I write in support of it. I’m posting here for two reasons:

  • Perhaps it’ll help someone else
  • Perhaps I’ll be able to find it in the future!

You can decide which is the more important reason.

Tweaking The Events Calendar

We chose to use The Events Calendar from Modern Tribe as our calendar tool.

Out of the box, it works well, but I wanted to display some information about the kinds of events we would show on the District Calendar and how to search for an event.

The obvious way to do that was to use the “Add HTML before event content” feature in Advanced Template Settings; unfortunately, when I added the information there, it not only appeared on calendar listings, it appeared when you chose a single event to display (and in that case, there was no search bar available, so it was really confusing!).

I found two ways to get the kind of display I wanted:

  • Override default-template.php by copying it from the-events-calendar/src/views/ to my-child-theme/tribe-events/ and then only calling tribe_events_before_html if I wasn’t displaying a single event; this has the advantage of making the information editable in the Events Calendar settings.
  • Add a new action for the tribe_events_bar_before_template hook that would display my information if and only if the search bar was going to be displayed. This has the disadvantage of requiring me to hard-code the information in my action.

I decided to go with the first choice, which also required me to be sure to use the “Default Events Template” instead of the “Default Page Template” in the Events Calendar settings.

Here’s the diff to create my modified default-template.php file:

@@ -17,7 +17,11 @@
 get_header();
 ?>
 <div id="tribe-events-pg-template">
-        <?php tribe_events_before_html(); ?>
+    <?php
+        if (!(tribe_is_event() && is_single())) {
+            tribe_events_before_html();
+        };
+    ?>
         <?php tribe_get_view(); ?>
         <?php tribe_events_after_html(); ?>
 </div> <!-- #tribe-events-pg-template -->

 
Posted in Life | Comments Off on Some Thoughts on the Toastmasters District 101 Website

Apple Watch Danger

I bought a loaf of La Brea Bakery bread today at Whole Foods, using Apple Pay on my Apple Watch.

I realize that’s not terribly noteworthy, but I’m annoyed at myself. The reason I bought that particular brand of bread at that particular store was to use a coupon I’d picked up – but, since I never had to open my wallet to make the payment, I forgot all about the coupon until I was most of the way home.

I’ll have to find a better place to stash my coupons. Maybe I could braid them onto the watchband?

Posted in Life | Comments Off on Apple Watch Danger

Goodbye, Sitemeter

I’ve been using SiteMeter since not long after I started this blog, back in 2000. It provided an easy way of counting visitors.

But today, I noticed weird out-clicks to “vindicosuite.com”, and a little research (thanks,
Jim McBee) showed that SiteMeter was the culprit, so they’re gone.

I apologize to anyone who got hijacked.

Posted in Life | Comments Off on Goodbye, Sitemeter

Strange message in /var/log/system.log

I updated one of my systems to OS X 10.10.3 today, and ran across this odd message in the system log:

NotificationCenter[293]: Layout still needs update after calling -[NCScrollView layout]. NCScrollView or one of its superclasses may have overridden -layout without calling super. Or, something may have dirtied layout in the middle of updating it. Both are programming errors in Cocoa Autolayout. The former is pretty likely to arise if some pre-Cocoa Autolayout class had a method called layout, but it should be fixed.

I hope somebody at Apple is looking at their logs….

Posted in Life | Comments Off on Strange message in /var/log/system.log

Rain!

It’s been a long time since we’ve had this much rain (about 1.6 inches since midnight). Here’s a photo and two very brief videos of Ross Creek near our house — it’s flowing, the ducks are happy, and there’s plenty of room left in it!

The ducks are happy

Video: Water under the bridge

Video: Water on the other side of the bridge

Posted in Life | Comments Off on Rain!

Saving my history – it’s needlessly complex

In 2004-05, my IBM colleague Sara Moulton Reger and I wrote a series of articles about “Needless Complexity” for IBM’s “Think Research” site as part of our work at Almaden Services Research.

The articles are no longer available on “Think Research”, so I’m reposting them here as a single PDF. Copyright, of course, remains with IBM.

Thanks to the Internet Archive for saving them in the Wayback Machine!

Posted in Life | Comments Off on Saving my history – it’s needlessly complex

I cut myself by ignoring Occam’s Razor

Occam’s Razor states that, all other things being equal, the simplest explanation is to be preferred. I wish I’d remembered that this afternoon.

We have just completed a major landscaping/hardscaping project – and as long as the driveway was being dug up, we decided to replace the 50-year-old galvanized pipe between the water meter and the house with brand new copper. The replacement went well, but after the plumber had left, I noticed that one of my toilets was refilling very slowly.

I ignored it for a couple of weeks, but finally I was irritated enough to Google the problem: “Fluidmaster 400A slow refill”. The top page in the results came from Fluidmaster, and it suggested several causes:

  • After several years of use, the seal of the 400A fill valve can become swollen, causing a slower fill.
  • The inlet of the valve or the water supply line is plugged with debris.
  • For new installations, if using a straight pipe as a water supply line, it may be extended too far inside the bottom of the valve shank.
  • For new installations, the valve cone washer may have been used with the wrong supply line.

This wasn’t a new installation, and the toilet was only a couple of years old, so I was pretty sure that debris was involved. Fluidmaster suggested removing the top cap from the valve and using the water pressure to clear debris from the valve. 30 minutes later, I’d finally managed to remove the top cap, only to discover that there wasn’t much pressure. Their explanation for that was “debris lodged deeper in the valve”, but trying to fix it would require draining the tank and getting water all over the floor, which I wanted to avoid, so I searched further.

HandymanWire gave me a great tip to replace only the top part of the valve (avoiding any need to drain the tank and get water on the floor), so I followed their procedure. 10 minutes later, I had the valve disassembled, ready to put on the top part of a new valve. But for some insane reason, I decided to test the pressure before putting the new valve’s guts on…and there was no pressure.

I drained most of the water from the tank and disconnected the hose leading to the tank. Then I cleaned up the water on the floor (oh, well) and turned on the water – still no pressure.

I disconnected the other end of the hose, right by the angle stop, and found a small pebble blocking the hose inlet. Removing the pebble solved the problem, and then I spent the next 20 minutes putting everything back together, including putting the old guts back on the valve.

In retrospect, the simplest explanation was a blockage in the water supply line – just as Fluidmaster (and Occam) suggested. If I’d tried that first, I would have been finished in 10 minutes (even with cleaning up the floor) — instead, I spent about 2 hours (including research and a couple of interruptions).

Oh, well; now I know a lot more about the insides of a Fluidmaster 400A valve than I did this morning.

Posted in Life | 1 Comment

Passport Day

Diane and I are back from Passport Day in Aptos. Three wineries visited, six wines acquired:

  • Pleasant Valley Vineyards – a beautiful backyard winery (well, it’s a 5-acre backyard) with an amazing stand of redwood trees. The wines were rather pricey (mostly north of $40), but quite tasty. They specialize in Pinot Noirs, which were very drinkable, but we really liked their Syrah and Zin, both of which were on the spicy side; we bought one bottle of the 2009 “Sean Boyle” Syrah, which will go well with a well-spiced steak.
  • Nicholson Vineyards – this was a slightly larger operation than Pleasant Valley, but still small and friendly. Their wines are made for drinking fairly soon and were considerably less expensive than Pleasant Valley. I wrote my tasting notes on their order form, which I seem to have left with them, so I’ll just list the wines we chose to purchase:
  • Finally, we visited Alfaro Family Vineyards, an even larger operation than Nicholson (they sold wines under three different labels, in fact). They had seven wines available for tasting, but we decided, in the interest of safety, to skip the Chardonnays; all of the wines were interesting, but we only picked up the Corralitos 2012 Syrah, which was pleasantly spicy, with a long finish and a relatively low price (hmm, I guess I can’t bring this one to a party now that I’ve written that!).

Three wineries in the space of 2.5 hours is a pretty brisk pace, and I’m sure I didn’t do Alfaro justice – I guess we’ll have to return.

Posted in Food and Wine | 2 Comments

I need rain

We haven’t had significant rain this season, and it shows. The hills are browner than usual; the fall garden is anemic; and there’s continuing talk about drought and water rationing. We even had someone ask at Torah study whether we should put the prayer for rain into the day’s service! Yes, we need rain.

But none of those reasons are why I need rain – I need rain so that I’ll stay inside and play work. When I look at my office, I realize that spending an hour or two reorganizing it would pay dividends – but that hour could be spent walking or hitting golf balls, and that’s what I do instead.

In fact, it’s a lovely day out right now. Bye!

Posted in Life | Comments Off on I need rain