Project suggestions for CS 5973 Fall 2005

The following list of project suggestions are not a complete list of possible projects. Please feel free to come up with your own project ideas as well! This list is intended to get your project ideas started. Also, feel free to take one of these projects and modify it. In addition to this list, Ray Mooney has a list of suggested ML projects.

Game playing

• General idea: Pick your favorite difficult game (for example, spades, hearts, backgammon, settlers of catan, checkers, go, monopoly, etc) and propose a learning approach to playing the game. Don't pick an easy game (like tic tac toe) as that would not be interesting.
• A specific example (This example is based on work by Dr. McGovern):
  • Suppose that you are fascinated by the card game spades and by the performance of TDGammon.
  • Propose to train a neural net to bid at an expert level for the game of spades
  • Write your spades software, your NN software.
  • Create many different features for representing the current hand
  • Gather training data from actual humans playing the game or from decent computer players (these exist nowadays)
  • Train your NN and analyze its performance. Where is it going wrong? Does it consistently underbid or overbid? Could your playing heuristic be at fault? Could your feature set not be expressive enough? Do you have enough training data? Is it consistent enough?
  • Show graphs of the performance over time (as you train). Show how it performs against various opponents.
  • Can you train an ML method to actually play the cards as well?
• A second specific example (This example is also based on work by Dr. McGovern):
  • Suppose you are fascinated by the card game hearts
  • Propose to train a reinforcement learning agent to play hearts
  • Specify a state representation (choose your favorite function approximation technique here)
  • Specify a reward function (perhaps points won/lost depending on how you view hearts)
  • Talk to Dr. McGovern to get a hearts simulator (or write your own if you really want to)
  • Write your player and compare it to other players (there are online hearts servers or you could write your own quick heuristic).
  • Analyze its performance: when does it win, can you shoot the moon, when do you lose? Do you always lose against certain players? Does it show any interesting features that a human player might? As with spades, do you have enough training data? Is it consistent enough?
• Other game playing ideas include writing a MOO bot that navigates through a MOO/MUD efficiently, a doom bot that plays doom well, or even a bot that generates the rules to a game on the fly and tries to make interesting games for a human to play. This last could be difficult!

Making money on the stock market

• The stock market provides a veritable wealth of data about how stocks go up and down. Much of this data is available online for free (provided you are willing to extract/format it yourself, I suspect).
• General idea: propose a specific learning approach on a specific area of the stock market and see how much money you can make!
• Specific examples (taken from a talk that Dr. McGovern saw):
  • Study a single stock (such as gold, silver, oil, or a single company such as GOOG)
  • Obtain the historical data for the stock and perhaps for related stocks (to provide state information)
  • Propose an ML method to buy/sell a prespecified amount of stock (e.g. train a neural net or learn a decision tree to know when to buy/sell or use an RL agent with reward being the money won/lost)
  • Break your historical data up into chunks of time and train on earlier pieces of time
  • Given an initial amount of money (say $10,000), evaluate your approach over a period of time that you did not train on
  • How much money do you have at the end of the time period? Do this for multiple time periods as any approach during the internet bubble is likely to look good and any approach during the depression is likely to look bad.

Weather data analysis

• Dr. McGovern has a enormous amount of severe weather data available through collaborations with the School of Meteorology. Projects in this area include classifying severe weather features including thunderstorms (single-cell, multi-cell, line, etc), automatically identifying severe weather features in simulated data (such as finding hook echos), and predicting the arrival of severe storms such as tornados.
• Neural nets have already been shown to work effectively for many weather applications
• Decision trees may be another approach that would work well here
• A specific project idea:
  • Train a decision tree to classify thunderstorms as single-cell, multi-cell, and line
  • Analyze the results and understand both why it makes the correct classifications but also why it is wrong
  • Using your tree, share the results with meteoroloigists and see if you learned anything suprising.

Constrained clustering and relational data

• Constrained clustering differs from regular clustering in that soft (or hard) constraints are provided for some of the data. For example, you can specify that two items should (or should not) be in the same final cluster. These constraints are sometimes difficult to provide but perhaps they can be derived using relations (such as block a is ON block b or player a sorted her cards with card 1 NEXT TO card 2). A project could examine the ability to use relational data as information for constrained clustering in a particular domain (movies might make a great example domain).