Categories
Uncategorized

LearnHub: Powered by Rails, Searches with Bing

This article also appears in Canadian Developer Connection.

Introducing LearnHub

learnhub_home_pageLearnHub’s home page.

If you’re a student applying to colleges and universities and are looking for help with the process, you should try LearnHub. Based in Toronto, LearnHub is a social learning network that helps students to prepare for standardized tests, assists with finding places to study abroad and provides career counseling. LearnHub’s site has hundreds of thousands of pages of free content, including the world’s largest bank of questions that appear in the GMAT and SAT standardized tests. The site has a large following among students worldwide, particularly in India, and has partnerships with 25 universities to recruit domestic and international students.

learnhub

With those hundreds of thousands of pages, LearnHub needed to provide a way for students to find what they’re looking for. They provide a search function, and it’s powered by Bing.

The people at LearnHub are part of that sector of Toronto tech that’s into Ruby on Rails, open source and founding startups. Founders John Philip Green and Malgosia Green are a husband-and-wife team who are known for building web applications for education and have been active members of Toronto’s tightly-knit open source tech community since the earliest DemoCamps. John caught Rails fever after trying it out and decided to rewrite a major application using it. The core development team of Wesley Moxam, Carsten Nielsen and Libin Pan are fixtures of the local Toronto’s on Rails scene; a gathering of local Rubyists doesn’t feel complete without them.

So what are they doing, using Bing?

Site-Wide Search

learnhub_dev_management_teamThe main room at LearnHub’s offices. Management are to the left, developers to the right.

In the beginning, they went with their first instinct, which was to use Google. “We launched in March 2008,” said co-founder John Philip Green, “and we needed to provide site-wide search, so we went with Google. We signed up, and for a few hundred bucks a year, we got a search function that covered about 5,000 pages. It seemed like a pretty big number, and we thought that would be more than enough to cover our site.”

They soon found that the results weren’t what they expected. “We weren’t getting good results. We’d use our site-wide search to search for something that we knew was in our site, and it wouldn’t show up in the results.” The same search would work just fine if you did it from Google.com, but not from their Google-powered search function. “The results just weren’t relevant, and we also had a limited number of queries,” John said.

learnhub_management_dev_teamThe main room at LearnHub’s offices. That’s management in the foreground, developers in the back.

LearnHub’s page count grew quickly and beyond the 5,000 pages covered by their arrangement with Google. “Going up to a bigger package was expensive;” John said, “it would have cost a couple thousand for 50,000 pages, and we were already at hundreds of thousands.”

“We could’ve gotten the functionality for free, but that’s only an option when you show ads in the search results, and the ads that showed up were for our competitors.”

learnhub_sales_teamLearnHub’s sales team.

There was another problem: Google’s site search returned its results as a web page. In order to make LearnHub’s site-wide search’s results page have the same look and feel as the rest of the site, they had to stick the Google results in an iframe. “And even then, what was inside the iframe didn’t match the rest of the page,” added John.

They started looking at other options for implementing LearnHub’s site-wide search, including running their own spider. “We really didn’t want to do that,” said programmer Wesley Moxam.

Enter Bing

wes_moxamLearnHub developer Wesley Moxam.

While looking around at search options, Wesley found the Live Search API, which is now known as the Bing API. “It was free, well-designed and spits out JSON,” he said. “Google requires a JavaScript interface or SOAP, and SOAP libraries in Ruby are painful.”

“It took a day to implement and get it up and running,” said Wesley, “The entire switch-over project happened over three days, with us working on it on and off, while we were doing other tasks. Best of all, we get consistent results – the results from the API are the same results you’d get if you just used the Bing site.”

“Bing’s API is simple and straightforward. You call it, you get the results, you take those results and use them how you like,” he continued. “It’s good. It’s hard to explain good software; good software is inherently simple.”

Here’s a screenshot of a LearnHub search results page for the search term “accordion” – and yes, the word appears on a handful of Learnhub pages!

LearnHub search results page for the search term "accordion" LearnHub’s search results page for the term “accordion”.

LearnHub have benefited from using Bing to power their site-wide search, and they’ve decided to share the wealth. Wesley’s working on refactoring the Ruby library he wrote to act as a wrapper for the Bing API and open source it for anyone to use. It should be available later this summer. He’ll announce it when it’s released, and I’ll announce it here.

The Bing API

Bing logo

It’s easy to harness the power of Bing in your applications, whether for desktop, web or mobile.

The first step is to get an AppID, which is a string that uniquely identifies you as a registered Bing application developer. Go to the Bing Developer Center, sign in with your Windows Live ID (which you can get for free) and follow the link to created a new AppID. You’ll be asked to supply some very basic information about your application and to review the Bing API’s Terms of Use. If you provide the information and agree to the Terms of Use (which I summarize in plain English below), you’ll get an AppID.

Once you have an AppID, you can start experimenting right away with the Bing API. All you need to do is start typing URLs with the format below into your browser’s address bar:

http://api.search.live.net/xml.aspx?AppID=<AppID>&query=<SearchTerms>&sources=<SourceTypes>

where:

  • <AppID> is the AppID assigned to you
  • <SearchTerms> are your urlencoded search terms
  • <SourceTypes> specifies the type(s) of search results you want. The different sourcetypes are explained in the table below:
SourceType Description Example Search Terms
Web Searches for web content accordion – returns web pages containing the term “accordion”
Image Searches for images on the web accordion – returns images of accordions
News Searches news stories accordion – returns news articles about accordions
InstantAnswer Searches Encarta online what is an accordion – returns the definition of “accordion”

convert 1.6 kilometres to miles – returns “0.9941939 miles”

sin(30 degrees) – returns “0.5”
Spell Searches Encarta Dictionary for spelling suggestions accordian – returns “accordion” 
Phonebook Searches phonebook entries accordions in Toronto – returns location results for “accordions in Toronto”
RelatedSearch Returns query strings most similar to yours accordion – returns results like “{piano accordion; button accordion; accordion store}”
Ad Returns advertisements to incorporate with results (use this to make money with you Bing-powered application) accordion – returns ads relevant to the keyword “accordion”

 

The default format for results is XML, and that’s the format you get when typing in API calls in your browser. You can also have the results returned as JSON or SOAP if you prefer.

You can find out more about the Bing API in the Bing API section of MSDN.

Bing’s Terms of Use, Explained as Simply as Possible

Here’s a quick explanation of Bing’s Terms of Use for those of us without a law degree. It’s adapted from the Bing documentation and provides a quick summary of what application developers using the Bing API must do and cannot do (besides the obvious "I promise not to use the API to plan a terrorist attack, run a drug smuggling ring or help the band Nickelback take forceful despotic rule of planet Earth").

What you must do:

  • You must display all the results you request. No filtering!
  • You must display your results in the context of a user-facing application or website.
  • You must display attribution to Bing in a manner compliant with our branding rules. Currently, you may determine the specific manner in which you display attribution. A link to http://www.live.com with the query echo is a suggested example.
  • You must restrict your usage to less than 7 queries per second per IP address. You may be permitted to exceed this limit under some conditions, but this must be approved through discussion with the folks at api_tou@microsoft.com.
  • If you interleave data from any source other than the API with data from the API, you must clearly

    differentiate the respective sources. (Yes, you can interleave Bing results with other data!)

What you cannot do:

  • You cannot use API results for search engine optimization (SEO). In particular, using the API for rank checks is explicitly prohibited.
  • You cannot display advertisements in positions other than the mainline and sidebar.
  • You cannot change the order of the results the API returns from a SourceType other than Web. (In other words, you can re-order results from standard searches for web pages!)

Bing Your Apps!

From there, the sky’s the limit. The Bing API is very straightforward and easy to use, it costs nothing to use it, and as someone who’s been using Bing as his default search engine since its beta period, the results it provides are great. Go forth and Bing your apps!

Categories
Uncategorized

One Million Pageviews!

This may not really be of interest to anyone but me and StatCounter, but earlier today Global Nerdy hit the one million pageview mark for 2009. I’d like to thank all you readers who keep coming back for more; I promise I’ll make it worth your while!

(And if you happen to run web ads, feel free to drop me a line. I have a readership!)

Here’s a screencapture of the my StatCounter page for Global Nerdy:

Screencap: StatCounter stats page showing Global Nerdy's 1 million pages for 2009.

Categories
Uncategorized

Iran is Taking Marc Stiegler’s Final Exam

Marc Stiegler

earthweb I met science fiction author, software developer and computer security guy Marc Stiegler at the first incarnation of O’Reilly’s Emerging Technology Conference in 2002, but I’d been acquainted with his work prior to that. I’d heard of his programming language called E and had read his science fiction novel Earthweb, whose plot could be grossly oversimplified down to the summary “Twitter saves the world” (it’s a little bit more than that, but I think it conveys the idea nicely).

Marc’s Final Exam

However, when I think of Marc, what comes to mind first is the final exam that he gave to students at his “Future of Computing” course and published online in 1999. In it, he posed a set of problems and asked them how a specific set of proposed web technologies could be used to solve them. The course and exam have a very strong sense of “technology trumps legislation”, an idea that was surfacing in the late 1990s.

In the exam, students had to pick 5 out of 11 problems that Marc posed and then explain how any combination of the following technologies could be used to solve them:

  • Unforgeable pseudonymous identities
  • Bidirectional, typed, filterable links
  • Arbitration agents
  • Bonding agents
  • Escrow agents
  • Digital Cash
  • Capability Based Security with Strong Encryption

(If some of these ideas are unfamiliar to you, don’t worry. They’re not important in the context of this article, and you can always Bing them.)

Here’s a selection of the problems posed in the exam. Remember, this exam is from ten years ago!

1) Searching for a decision analysis tool on the Web, you find a review in which the reviewer raves about a particular product. You buy the product and discover it just doesn’t work. You desire to prevent this person’s ravings from harming anyone else–and you desire to prevent the product from disappointing anyone else.

4) You start receiving thousands of emails from organizations you don’t know, all hawking their wares. You want it to stop, just stop!

5) You wish to play poker with your friends. They live in Tampa Florida, you live in Kingman. This is illegal in the nation where you happen to be a citizen. You want to do it anyway.

6) You hear a joke that someone, somewhere, would probably find offensive. You wish to tell your precocious 17-year-old daughter, who is a student at Yale. The Common Decency Act Version 2 has just passed; it is a $100,000 offense to send such material electronically to a minor. You want to send it anyway–it is a very funny joke.

7) Someone claiming to be you starts roaming the Web making wild claims. You want to make sure people know it isn’t really you.

The Final Question

The most compelling question on the exam is the final one. It required a far more extensive answer than the other ten – so much more extensive that Marc actually suggested that it might be better not to answer the question in the exam, but to at least think about it:

But…if you can answer Question 11 in your own mind, even though you choose not to write up that answer for this examination, then a most remarkable thing will happen: you will walk out of this class with something profoundly worth knowing.

Here’s that final question:

11) You live in North Korea. Three days ago the soldiers came to your tiny patch of farmland and took the few scraps of food they hadn’t taken the week before. You have just boiled the last of your shoes and fed the softened leather to your 3-year-old child. She coughs, a sickly sound that cannot last much longer. Overhead you hear the drone of massive engines. You look into the sky, and thousands of tiny packages float down. You pick one up. It is made of plastic; you cannot feed it to your daughter. But the device talks to you, is solar powered, and teaches you how to use it to link to the Web. You have all the knowledge of the world at your fingertips; you can talk to thousands of others who share your desperate fate. The time has come to solve your problem in the most fundamental sense, and save the life of your daughter.

The final question really stands out. Unlike the other questions in the exam, this one really pulls at the heartstrings, and it sparked a lot of discussion among geeks back around 1999 and 2000, in settings both online and real-life.

Iran and the Final Question

If you follow the American news cycle, the mental distance between North Korea and Iran is a short one; both are countries in the “Axis of Evil” (a term invented by a Toronto guy, by the way) run by repressive regimes and working on their nuclear weapons capabilities. What if we changed the final question’s setting from North Korea to Iran?

Unlike North Korea, Iran’s people have access to technology and communications with the outside world (there’s a recent Daily Show segment in which Jason Jones finds people in Iran who know Jon Stewart’s George Bush “I’m the decider” schtick). They don’t need to have Marc’s hypothetical iPhones delivered to them in care packages; they have things like Twitter and YouTube at their disposal. So I propose another slight modification to the final question: What if we changed the hypothetical hardware into actual working software like Twitter and YouTube?

(It’s another “software, not hardware, is really the trick” situation. Just as we found out in Terminator 3 that SkyNet was really software, it turns out that what might save Iran was social networking software, not portable internet-accessing hardware dropped by parachute.)

With my two suggested changes, it becomes very apparent that we’ve moved from theory to practice. The people of Iran are taking Marc Stiegler’s final exam, and they’ve picked its most difficult question.

Let’s hope they pass.

Categories
Uncategorized

Scenes from May’s Metro Toronto .NET User Group

Better late than never! Here are a couple of pictures I shot at the Metro Toronto .NET User Group in late May, where I presented my walk-through of ASP.NET MVC, Canada’s Next Top Model View Controller.

Colin Bowern opened the session with some quick announcements about upcoming events as well as other .NET user groups in the Greater Toronto and surrounding areas:

metro_toronto_dot_net_ug_1

He then introduced me, and I got started with my presentation. I did a quick walkthrough of the basic concepts behind MVC (that is, the Model-View-Controller design pattern), after which I introduced a special guest who I brought along with me: Paul Doerwald. Paul’s a Ruby on Rails developer, and he gave the audience a quick demonstration of building a Ruby on Rails project from scratch:

metro_toronto_dot_net_ug_2

Bringing in a Rails guy to speak in front of a .NET crowd was a little unexpected, but I thought it was important to show them the inspiration behind ASP.NET MVC, whose creators acknowledge Ruby on Rails’ influence on their framework. I think that they benefited from this outside perspective, and it worked on Paul as well – he was impressed by the size of the crowd, the nice settings (the Metro Toronto .NET User Group has a nice arrangement to use the conference rooms at the Manulife office building), the extent and organization of all the .NET user groups in the Toronto area and even the male-female ratio (while the women were still vastly outnumbered by the men, the percentage of women at the User Group was still high in comparison to some open source gatherings).

Much of my presentation was a walk-through of building the basic structure of the NerdDinner application featured in the book Professional ASP.NET MVC 1.0 and online at NerdDinner.com, with plenty of additional commentary by me, explaining in further detail why things were done a certain way. I encouraged the audience to download the free chapter from the book and actually build the application themselves; after all, the best way to learn is to do.

I had a wonderful time presenting in front of the very attentive and appreciative crowd at the Metro Toronto .NET User Group and would like to thank the audience for watching, Paul for helping out and the organizers for inviting me. I’d love to do it again sometime!

As for ASP.NET MVC, watch this space for more articles and code examples!

Categories
Uncategorized

Toronto Coffee and Code This Friday!

coffee_and_code_may_29_2009_1The scene at the big communal table at the May 29th Coffee and Code.

There’s a Toronto Coffee and Code this Friday! For details, see the Coffee and Code blog.

Categories
Uncategorized

Living in the “Hooray!” Zone

Yes, the demands and schedule of my job as Sith Lord at Microsoft have kept me quite busy, but it doesn’t matter because I live in the “Hooray!” zone, as shown in the Venn diagram below:

Venn diagram showing the "Hooray" zone as the intersection of "What we do well", "What we want to do" and "What we can be paid to do"

For more information, see the LifeHacker article titled The Road to Happiness in Your Work Lies in the Hooray! Zone.

Categories
Uncategorized

Silverlight on the Silver Screen: Thursday, July 9th in Toronto

Movie theatre with the Silverlight logo projected on the screen

The covers will be coming off our next generation of user experience tools and technologies on July 9th. That’s when Microsoft will be unveiling Silverlight 3, which gives you the all the goodness of RIA (Rich Internet Application, although you can use Silverlight to make desktop apps as well) with out the PITA (Pain In The Anterior regions).

To help promote Silverlight 3, we and our pals at ObjectSharp will be co-hosting Silverlight on the Silver Screen live at the Scotiabank Theatre (259 Richmond Street West, at John Street) in Toronto on the morning of Thursday, July 9th from 9:00 a.m. till noon (and yes, the event is free). The ObjectSharpies are early adopters of SIlverlight and have forgotten more about it than most people will ever learn. As seasoned pros, they’ll share their stories and wisdom about the next-gen version of Silverlight, as well as associated tech such as Expression Blend, SketchFlow and the touch technologies in Windows 7.

Joining them will be my friends from the DPE team, who’ll be there to talk about the opportunities offered by Microsoft’s “UX3” platform – they’re a great way for your development team and business to stand out in the crowd and give your customers a great user experience.

And yes, the accordion might make an appearance.

As I said earlier, the event is free and takes place on the morning of Thursday, July 9th. All you have to do to attend is register at the Silverlight on the Silver Screen page!

Movie poster-style banner for "Silverlight on the Silver Screen"