undesigned

Why is it that students so often claim “I am no good at math”? Here is one theory:

1. In primary grades the major emphasis is on recall of basic math facts (e.g. 3 + 5).
2. Many kids aren’t wired for simple recall (they aren’t good at memorizing). I’m not.
3. Commonly used instructional approaches don’t give the immediate feedback essential to developing simple recall (e.g. do this worksheet which will be graded later today or tomorrow).
4. Because of the heavy emphasis on simple recall kids perceive that it represents all of math.
5. This Simon-Says approach to math instruction often continues throughout a student’s public education.
6. By the time students are given the chance to engage in other types of mathematical thinking, they have developed a negative self-perception that pre-disposes them against trying or learning math.

Posted on April 22nd, 2008 by joel
Filed under: math education | 1 Comment »

I recently moved from hostgator to slicehost. I signed up with hostgator because it seemed to be a cheap place ($10/mo) to play with rails. It turns out that hostgator doesn’t really do rails (they offer it via cgi, not even fastcgi). They didn’t allow me to install things like the Send2Wiki perl module or give me access to execute scripts from PHP either.

With slicehost I get a VPS for $20 / mo. I’ve never used a VPS before but have run my own web servers, so it seems worth a shot. In the process of getting it set up I’ve learned more about DNS, iptables, fastcgi, etc than before, so that is fun, but also a time drain. Anyway, now I’ve got the access I need to set up Send2Wiki and run rails apps.

After hearing Justin rave about WPMU, I thought I would give it a shot. WP normally installs easy, so I expected the same. It would have except I wanted to install WPMU in a subdirectory. My Googling gave warnings against doing so, but I shrugged them off and forged ahead and tried to hack it to get it to work. Bad idea. It may be possible to do, and I got it to almost work, but certainly it doesn’t fall within the scope of famous 5 minutes install. So I’ve backed off and returned to the single user version.

Posted on April 21st, 2008 by admin
Filed under: web development | No Comments »

A couple of years ago I wrote the NLVM Application using Java Swing 1.4. It incorporates the Interactive Math Applets from the NLVM Website. The challenge was that those applets were written based on framework built back in the bronze ages when Netscape on Mac 8 was used heavily in the schools. At that time I Googled around to figure out how to mix Swing and AWT. I decided to ignore the warnings because of the large code base and eventually got it to work.

This past week I began updating the NLVM Application to support switching between multiple languages. On the screen that displays applets I added a JComboBox below the applet containing a list of languages that a user can select. Everything seemed to work fine until I noticed that once I clicked on the list box the applets stopped updating. After much pain and suffering and digging through the Swing source code I figured out the problem:

When JComboBox gets focus it (via javax.swing.plaf.basic.BasicPopupMenuUI.grabContainer) calls addMouseListener, addMouseMotionListener, and addMouseWheelListener on the applet, which are implemented in java.awt.Component. This is where the problems begin. All of those methods contain an innocuous line:

newEventsOnly = true;

There is the evil. The effect of this flag being set bears fruit in java.awt.Component.dispatchEventImpl where there is a set of nested if statements that begins:

if (newEventsOnly) {
if (eventEnabled(e)) {
processEvent(e);
}
}
...

The result is that none of the old AWT events (that the applets depend on) are sent after one of those listeners is added and the flag is set. It turns out, there are more methods that set the flag as well: addComponentListener, addFocusListener, addHierarchyListener, addHierarchyBoundsListener, addKeyListener, and addInputMethodListener, but the JComboBox never calls those methods.

My simple hack was to override those addXXXListener methods in my base class to prevent the calls from reaching down into Component where the flag gets set. I did this by writing a new J14Applet base class which extends java.applet.Applet and extending my former J10Applet base class from J14Applet instead of Applet. I’m sure this may have side effects but it seems to work fine for now.

As to why JComboBox was adding those listeners in the first place? I think it is so it can handle mouse messages when it has the focus but the mouse is not over the component, so I have defeated that behavior, but I can live with it. It is not as clean a hack as I would like because I want to be able to compile the applets for running on machines that don’t have a 1.4+ JVM. It would be nice to be able to have my Swing app dynamically add those methods like you can do in Ruby, but I don’t quickly see how to do that. For now, I will just change what J10Applet extends depending on what target I’m compiling for.

Posted on April 3rd, 2008 by joel
Filed under: java | No Comments »

Every time we update the NLVM website like we did a few weeks ago we receive email from people that are no longer able to access the applets. Often times the causes are somewhat mysterious. Some of the problems are caused by proxy and browser caching; some of the updated files arrive at peoples’ browsers and others do not. After a few days the problems seem to work themselves out.

Other times problems occur because we broke something and we didn’t catch it in our testing. This is aggravated by browser incompatibilities. Because old hardware and software tend to hang around schools longer than other places, strange things show up. Right now I’m trying to track down some of those types of issues.

A few years ago when I was more actively developing the NLVM I used to keep old machines around so I could test old browsers. One of the reasons multiple machines were needed was that I couldn’t find an easy way to run multiple versions of Internet Explorer on the same machine. Yesterday I googled to see if there is anything new out there to help with this issue. I was pleasantly surprised to find a utility by tredosoft that allows you to install multiple versions of IE on your PC. Thanks tredosoft! Unfortunately after installing the multiple browsers, I’m still not able to see the reported problem even when running on the same browser.

Posted on December 13th, 2007 by joel
Filed under: interactive online math, web development | No Comments »

Here is the updated OER Recommender White Paper.

Yesterday we released the OER Recommender system that I have worked on. There are still many things that could be added or tweaked, but it does something useful already so out the door it goes! I’m concerned that we are calling it a recommender as the “recommendations” it currently provides are not specific to a user, they are related resources generated via a content-based approach. The proposal and plans are to make it a recommender based on user profiles. See my previous post for details about where it is intended to go.

The Problem. The William and Flora Hewlett Foundation, the National Science Digital Library, and other large organizations have made large investments in the development of electronic resources that can be used for teaching, learning, and research. They are keen on finding ways to increase the impact of their investment. One way to do that is to create tools that make it easier for poeple to find resources that are useful to them. One approach to solving this problem are search and browsing tools that help people find good resources such as NSDL, OCW Finder, and of course Google. Recommender systems approach the problem by helping bringing resources to peoples’ attention without having them to go look for them specifically Google’s Adsense technology and Amazon’s recommendations are two of the most common examples. Some of the challenges with general search tools such as Google and even more focused ones such as NSDL’s main search is that they cast too wide a net and you get back resources do not match. On the other hand, while most collections of open courseware and digital libraries provide search, it is nice to be able to search across repositories at the same time to find the best resources wherever they reside.

The Vision. In creating the OER Recommender we set out to create a service that would help people find relevant open education resources. The first step in this was to create an automated process for clustering related resources. The hope is that presenting links to related resources to users when looking at a resource could help people stumble upon resources that are relevant to them even though they might have not been actively looking for them. We could tune this service by monitoring what resources people visit, share, rate, tag, and otherwise pay attention to. We could use this attention metadata to create profiles of what people’s interest are. The profiles could be used to push recommendations to people even when they are not browsing perhaps via email or other means.

Exploration. Since learning about it I have felt that the recommender would be one of the more interesting folksemantic tools to work on since I initially thought to pursue a latent semantic analysis approach. I had heard about LSA from my exposure to Andy Walker’s and Art Graesser’s work. I have also had a number of interesting discussions about clustering with Adele Cutler about her work on Random Forests. I got a hold of the Handbook of Latent Semantic Analysis and explored the LSA online resources I could find as well as the lsa module for R. After spending significant time reading and playing I came to the conclusion that it would likely more complex to implement and computationally expensive than I wanted. I also concluded that everything that I would need to do to implement a standard Term Vector Model approach could be used to later implement an LSA approach. During my exploration, I came across a number of useful online resources and books including Gerard Salton’s work.

An Explanation for My Mother. I’ve already been asked by Shelly to explain the implementation of the recommender in language that my mother would understand, so I’ll give that explanation first. We use the folksemantic feed harvester to gather information (metadata) about OERs into databases. For each pair of resources that are related enough, the recommender uses the titles, description, and tags to calculate a score indicating how related they are. Recommended resources are the ones scored to be the most similar. The similarity of two resources is based on an automated analysis of the words in their metadata. Got that Mom?

We use four phases to arrive at recommendations: (1) parse metadata, (2) calculate local term weights, (3) calculate global term weights, (4) calculate similarity scores.

Parse Metadata. Using a string tokenizer we break metadata text into terms. We throw away stop words (common words such as ‘and’, ‘is’, ‘of’ that do not add meaning). Next we convert terms into their stems using a common algorithm; for example ‘running’, ‘ran’, and ‘runs’ all get collapsed into ‘run’.

Calculate Local Term Weights. A local term weight is a measure of how important a word is for describing a document. The more frequently a word appears in a document, the more important it is… up to a point. Of course, where a word appears in a document is probably a good indicator of how important the word is as well. For example if a word appears in a title, it is probably more important than if it appears in the body. One more important factor to consider is document length (total number of terms in the document); all other things being equal, the longer a document is, the more times a term will appear in it. So we normalize term frequencies using the document length. One last issue relates to filler words.

To calculate local term weights OER Recommender uses a function that looks like this.

The x-axis represents term frequency and the y-axis represents the local term weight. As you can see, the more time a term appears in a document, the more weight it is given. However after reaching a threshold, it plateaus. I think this is especially important in situations where document creators might be trying to game the system. The formula used is:

ltw_i = 1 / (1 + (e^(.0044)dlen) .7^{(fti - 1)})

ltw_i is the local term weight for a given term in a document.

dlen is the total number of terms in the document.

ft_i is the number of times a given term appears in a document.

Note that the constants .0044 and .7 in the equation determine the shape of the curve. I chose those values based on their use in the MeSH system. As explained there, the values should be tuned to your data set using an empirical approach. I played with the parameters some and the values they used seemed fine, so I adopted them.

Calculate Global Term Weights. Global term weights are a measure of how important a word is for distinguishing documents within a collection. The more documents a word appears in, the less value it has for characterizing documents. For example, the term ‘USU’ appears in every metadata record for resources in USU’s OpenCourseware. As a result, it has no value for characterizing clustering resources. The term ‘USU’ becomes in a sense, a stop word, similar to those thrown away during the parse phase. To calculate the global term weight for a term, the number of documents that it appears in are counted as well as the total number of documents.

To calculate global term weights OER Recommender uses a function that looks like.

The x-axis represents the number of documents that a term appears in. The y-axis represents the global weight assigned to the term. The formula used is:

gtw_i = log (D/df_i)

gtw_i is the global term weight for term.

D is the total number of documents.

df_i is the number of documents that the term appears in.

See Mi lslita’s explanation of Term Vector Theory for details.

Calculate Similarity Scores. Once local term weights and global term weights are calculated, we are ready to create recommendations. To create recommendations for a document we first find all documents that have any of the same terms in it. It turns out that this can be a very large number of documents (e.g. 40,000 in our system). For each pairing of the document being considered and a document with matching terms we calculate a similarity score. Because calculating 40,000 scores can take a long time (about 15 seconds in our current system), we shorten the list to consider to 200. We do this by sorting the pairs according to the number of overlapping terms. To calculate the similarity score for a pair of documents, we sum over all of the terms that they share in common. The contribution from each term is a combination of the local term weight in the first document, the local term weight in the second document and the global term weight. Because in OER recommender, each feed can be considered a separate collection, we calculate global term weights for each of the feeds and use them in calculating the similarity score.

To calculate the contribution of an individual term to the similarity score, OER Recommender uses:

sst_i = (gtw_1i)(ltw_1i)(gtw_2i)(ltw_2i)

sst_i Is the contribution to the similarity score for a given term.

gtw_1i is the global term weight from the feed that the first document is in.

ltw_1i is the local term weight from the first document.

gtw_2i is the global term weight from the feed that the second document is in.

ltw_2i is the local term weight from the second document.

See NCBI’s explanation of how MeSH calculates Related Articles for details. Once OER Recommender has calculated similarity scores for the 200 documents, it sorts the documents by those scores and stores the top 10 in a database.

Displaying Recommendations. Now that the recommendations are stored in a database, displaying them to the user is straightforward. We have created a Greasmonkey script that requests recommendations for each web page that a user browses. If OER Recommender returns any, the script inserts HTML for the recommendations into the web page. The eduCommons team is building a plone tool so that anyone running eduCommons can turn on recommendations. That way users won’t have to have install the Greasemonkey script in order to see recommendations on the eduCommons website. The OER Recommender site publishes the XML format it returns so others could add recommendations into their websites.

Getting Resources into the Recommender. Currently OER Recommender only provides recommendations for URLs that it has metadata for, so in order to get recommendations, you need to register a metadata feed. If you have an OCW site that provides an RSS feed, you can do that at OCW Finder. If you have an OAI feed or an RSS feed for learning objects or other OERs, you can add your metadata by sending a feed title, URL, and display URL (home page of the repository) to oerrecommender AT cosl DOT usu DOT edu.

Future Directions. There are many things left to be done: (1) adapt recommendations by monitoring which recommended resources people click on and how long they stay at recommended resources, (2) provide links to more recommendations (so users can see more than the default 5), (3) provide a way for people to indicate when a recommendation doesn’t work, (4) provide a way for people to register and login, so they can receive personalized recommendations (perhaps via email), (5) create a process whereby recommendations can be updated without going through all the documents in the database, (6) add functionality for retrieving recommendations for arbitrary web pages (recommending on demand) perhaps via a bookmarklet, (7) create whatever Greasemonkey-like add on is supported on IE. We also plan to integrate the recommender with the aggregator and feed reader to provide recommended news items based on the attention people have previously paid to news articles.

Wow! I realize that this is a long post, but it is not a simple topic. Hopefully the explanation is useful.

Posted on August 23rd, 2007 by joel
Filed under: information retrieval, web development | 4 Comments »