In order for this site to work properly, it uses cookies and javascript.
You can find more information here.

In order for this site to work properly, it uses cookies and javascript.
This site also tracks visits anonymously using cookies. Click 'agree' to confirm you are happy with that.
You can find more information in this site's policy.

Using Reinforcement Learning To Learn To Play Tic-Tac-ToeAbout a year ago I set myself the goal of writing an algorithm that could learn to play tic-tac-toe. I didn't want to tell the algorithm what the rules of the game are, nor did I want it to try and use some kind of calculation to look ahead at possible ...

The best opening move in a game of tic-tac-toeAs part of a machine learning project, I had to understand tic-tac-toe better, and so I have written an algorithm which a) finds all the possible unique games and b) gathers statistical information about those games. Based on Wikipedia's tic-tac-toe ...

Technical Project Management

manager is the name I have given to my new project management tool. Think Gantt chart. Think Microsoft Project. Then think the real world. Team members have different charge profiles - rates can change during projects. Team members can only provide given effort if they are partially assigned to other projects. Updating plans is largely manual, and tracking of hours spent takes a lot of the project managers time. Once underway, the plan becomes a reality, and things quickly change. The manager is continuously replanning, which is rather an administrative task. He wants to be managing, not administrating.

According to Prince 2, a project management methodology, the job of the project manager is to identify and then manage problems. He does so using thresholds. When a given threshold like overall cost, steps over a boundary, he alerts the next level and together they manage the problem, be it with added resources, reduced scope, increased budget, etc.

Getting information on when thresholds are over stepped, is again largely a manual and administrative job. In fact project cost analysis is also largely manual, and playing with scenarios, like removal of a feature (a set of related tasks) is largely theoretical, because without the right tools, it cannot take things like team members effort profiles, or holidays into account. Wouldn't it be great if there was a tool that took all this administrative stuff away from the manager, automated it, and just alerted him when there were problems?

Well... now there is one. Or at least there will be soon, since I'm writing it. Once its finished, it will allow you to plan your use cases (including capturing them). Next, it allows you to split use cases into features (see FDD). If use cases are thought of as vertical processes, then features are the horizonal functionalities. For example, a use case would describe an entire business process as the user interacts with the application. A feature, describes some functionality that might be used by one or more use cases, for example a dialog box. The idea is that a feature is something that a developer can develop in around a weeks worth of work. A feature is analagous to a task in eXtreme Programming (XP).

After capturing features (in terms of a design document), you define how long you plan the five steps of a feature to take - design, development, testing, review and rework. You assign team members, who you have previously added to the model, with relevant roles, effort profiles and cost profiles. You also relate the features to each other in terms of their dependencies. What needs to be built first?

Now, the nice part is that we have all the information to automatically plan the project. Exactly how, is a bit complicated, and commercially sensitive, but it can be done.

At this stage, you can add mile stones and thresholds. Thresholds might be projected cost over run, time over run, etc. At this stage, you can also play with your model. If it turns out too expensive, you can remove features at the click of a button and see what cost savings you can make. You cannot simply put a cost on a feature or use case. It really depends upon when it is done and by whom. So you need to consider the entire plan. This tool is perfect for the job.

Now, when ready to start you publish a report. It emails your staff the features that they should start working on. The report, is a website where all the documentation is published, and accurate and up to date plans are available. As they complete work, the team members log their hours into the application.

Comparing the actual effort to the planned effort, for each team member, you can work out a "velocity", which is the ratio. This allows the tool to predict where you are headed, and to replan the future work. At the point when a threshold is over stepped, the manager gets an email. He can then play with the model to work out a way out. Dropping features or applying for more staff or more budget are easily analysed and the results can be simply displayed to the stakeholders.

Oh, and it tracks the critical path too. Here is a screen shot (click it to see an enlarged version). Watch out, it will soon be available for download at www.maxant.co.uk.

"Replace an enterprise information system that has been developed over the last ten years, to allow it to become stable on a modern architecture, to allow two new business units due to a split in the business to develop along their own paths, and to introduce new features that will take the business into the multinational domain."

Inception

The business wants to know how much such a system will cost. Although its vital for the entire business to be able to continue, it does make sense to get a grip on the costs of such a system, because with a vision like that, they are likely to explode. So the business has approached the IT department and asked for a conceptual phase to be carried out that includes analysis of requirements, conceptual analysis, migration plans and cost estimates.

Great, three months ago we started on the technical side. No requirements at this stage, other than the vision. We managed to put together the architecture though, with big caveats with regards to realisation based on actual requirements.

Hang on... shouldn't we have stopped before writing about the architecture and waited for requirements? Well the project manager insisted we keep to the deadlines, so perhaps we could "put something together"? His budget, he can spend it how he likes (I get paid by the hour).

Two months ago, we completed the migration document. Again, because of a lack of requirements, it was largely conceptual, with lots of pretty flow charts to make migration decisions in the future become easier.

Two weeks ago, I went to my weekly meeting with the overall project manager, and he passed me seven sheets of "requirements". They were notes made by the 4 business analysts over a 6 week period. "That, Ant, is all we expect until the end of the year. I realise were late with them, but based on all your good work to date, I trust we shall have the estimates by 1st December?"

Elaboration...

Well, this one was actually dead easy to deal with. "Sure, I can tell you right now. It will cost you nothing. Because with what you have given me, I don't have enough information to actually build anything."

That gut reaction wasn't voiced, instead I took it to our account manager to get him to manage the customer a bit. His vast experience in the clients business sector was actually excellent. We went through the document, and decided 70% was out of scope as it belonged in existing systems. Some of it was light years ahead of what the business would be able to support, in terms of them being able to supply accurate data that these systems could run with. We decided the cost was in the range of 1 million, and 50 million. Its just a shame we are required to estimate within 30% so that the business can make a go/no go decision, after which we are bound by the 30%.

Transition

The outcome is that we shall have a workshop with the business analysts to be able to determine what is in scope and what is not. There after, we shall write some use cases by interviewing them (hang on, isnt writing use cases their job?). After that, we will grade the required functions with respect to each other, in terms of there technical complexibility and size.

The business can then apply their own weightings based on priorities, to decide what parts should be investigated further, and implemented like real projects, as opposed to these fairy projects from cloud-coo-coo land.

Lesson Learned

In the real world, we gather complete requirements that could be the basis of a legally binding document used to check off the deliverables. UAT is run against these requirements. If they are met, we have delivered successfully (in terms of funtionality - see the kitchen paper).