My EclipseCon (2011) Experience

It was great to go to EclipseCon finally this year, after using Eclipse IDE for 6+ years and working on Eclipse RCP application for around 2+ years now. It was also good to meet and go to sessions from some of the well known names in this field (Jason Van Zyl, Eric Clayberg, Dan Rubel, Paul Webster, Tom Schindl, Kai Toedter, Patrick Paulin etc.,).

For those who could not go, below are some of the notes from some of the sessions I went to (in no specified order):

1) Tycho - Tycho provides a way to build Eclipse plugins. PDE ant build now does the same, but Tycho promises to be much easier than PDE build. Sonatype gave a good tutorial on Tycho and also showed how to automate it in Hudson. They also promised that they will release a online book on Tycho soon.

2) Jubula - Jubula seems like a great functional testing tool and it is from the same company (BredEx) who developed GuiDancer. The cool thing with Jubula is we can start writing test case scenarios before the actual development happens. This would enable testers and maybe product owners to start writing acceptance tests before development team starts working on a task. When development is done, we can start mapping components in use case scenario to actual components in the product. Also, each scenario (or sub steps in a scenario) can be created independently and thus can be reused by multiple tests. So, if a component changes in the screen, it is enough to just update that one sub-step and all test cases are updated automatically, since they just reference this step. This makes it very modular and easy to maintain. They gave a good tutorial on Jubula during the session. They are also planning to hold a webinar on Jubula soon. They also said that Jubula tests can be run in command line and can be automated to run via any CI system. Jubula can be downloaded now, but will be official in the upcoming Indigo release.

3) Chronon - This is a time traveling debugger. The idea is someone (say tester) comes across a bug and then he can send the recording (which records every line of code executed) to developer. The developer can play back the recording using time traveling debugger and can see the actual values of variables used in that execution. Also, it makes it easy to see execution path of multiple threads, which is hard to see via regular debugging. This is a very novel idea and won the best hot products showcase award for 2010. Of course, I voted for it too.

4) BIRT 360 Dashboard and BIRT Data Analyzer - We use open source BIRT as an eclipse plugin in our RCP application to produce PDF/HTML BIRT reports. But, it is not very interactive. BIRT 360 gives dashboard like functionality to end users. It would be very useful for management/executives to see what goes in their company (for example, sales report by region etc.,) at a high level easily instead of reading a big report. BIRT data analyzer goes one step further and allows users to put their own analytics in their dashboard/reports. With data analyzer, users can pick any value from the data cube in the report and can show it in any way in their report. But, both BIRT dashboard and data analyzer are not in open source and are only available in Actuate's commercial BIRT software. Also, both of them are web based applications and are not available as RCP plugins to be used by desktop applications.

5) e4 - There were many talks about e4 in EclipseCon. e4 mainly gives new way for styling and provides flexible programming model. e4 is not backward compatible with 3x version of the Eclipse plugins. However, they provide a compatibility layer which helps in running 3x plugins in e4 environment. But, still there could be issues (with actionSets, menus, menuContributions etc.,) even with that. e4 does not support activities/capabilities yet. They said it might be supported in 4.2. When developing new eclipse plugins/rcp applications, it makes sense to use e4. But, I'm not convinced about using e4 on existing applications, as the migration could take a while, especially if the code base is large.

6) Q7 - Another functional UI testing tool. It is created by XORED (a Russian Eclipse RCP consulting company) and there were many participants from XORED in the conference. Good thing here is test scripts are in ECL (Eclipse Command Language), which makes it easy for testers to read/edit the test cases. They plan to add support for BDD soon, which would make it even better. They have Q7 maven plugin which could be used to run Q7 tests via maven using Tycho. So, this could automated via CI. However, it does not seem as easy/flexible to use/intuitive as Jubula.

7) Nexus Pro - Sonatype gave a talk on how to improve onboarding of new developers using Nexus Pro. Developer Onboarding feature is in Nexus Pro and automates the normal tasks done by new developers (Downloading, Installing Eclipse, Eclipse Plugins needed, checking out source code etc.,). Basically, we just have to send a jnlp link to the new developer and when he/she clicks on that link, it guides the user to install everything needed. It does not have support for setting up other tools though. They plan to add support for downloading/setting up webserver, installing maven etc., in future. It seems like Yoxos does something similar to. You can look at Yoxos OnDemand here. Nexus Pro also supports Eclipse update sites and p2 bundle repositories. If you need to use Nexus to host Eclipse bundles, then this would be handy. If you need to know what is the difference between Nexus Pro and Nexus OpenSource, you can see it here.

8) Orion - It is aimed at making Web-based development easier in the Web. It is basically a set of linked web pages and not an IDE in a browser. File navigation and editing happens from within a browser. It looks pretty neat, but I think it needs more features before it can replace the desktop based IDE.

9) Commands - Paul Webster gave an interesting talk on 10 most patterns in using Eclipse 3x commands. The slides are available here. The session was very informative. He also gave a talk of e4 commands and slides are available here.

10) SWT 3.7 - There are many fixes/minor improvements in 3.7, but the main feature that is new is Touch and Gestures. A touch is a change in state of a finger on the input device and a Gesture is interpretation of touches. It provides events on touch/gesture just like we get Key/Mouse events. This is very cool and opens the door to make Eclipse RCP applications very interactive.

11) Keynote sessions - They had three keynote sessions - one on IBM's Watson project, one on Java Renaissance and one on Apache Hadoop. The one on Java Renaissance was interesting and speakers from IBM and Oracle gave the talk. They confirmed that Java 7 release date is July 28, 2011 and Java 8 is targeted for end of 2012. Java 8 focuses on making Java modular (Project Jigsaw). They also spoke about making the new modular framework play nicely with OSGi. They also gave some insights on what could come in Java 9. Some of the things they hinted are supporting massive multi-cores, multi gigabyte heaps etc. in Java 9.

12) Eclipse 3.7/3.8 - Eclipse 3.7 code named Indigo will come out in late June just like all previous Eclipse releases. It may/may not have support for Java 7 in that. However, they said that 3.7.1 will have support for Java 7, if the initial one does not have it. Also, they confirmed that Eclipse 3.8 is code named as Juno.

Finally, here are some stats on who attended. There were lot of attendees and it was very diverse too. They showed these stats (and other fun stats like how much beer the attendees drank etc.,) during the closing session. I wish they had provided details on how many people actually attended.

50% of attendees are from US - 25% from California and 25% from rest of US (Totally from 31 states)
10% of attendees are from Canada
40% rest of the world - Germany, France, UK, Sweden, Russia and Bulgaria had sizeable attendees
Totally, there were participants from 29 countries.

GMaven Goodness

I recently had a requirement where I wanted to get a substring of artifactId and then use it in another plugin's execution. I thought it would be easy, but it was not as easy as I thought. Initially, I had couple of options in mind:

1. I could declare a property directly in my POM which has the substring of artifactId and then use it. But, the problem is we have so many projects and each of the projects have to declare this property. Also, we need to make sure that any new project in future declare this property too. Since, this is too manual, I was looking for an automated solution.
   
2. I could write my own small maven plugin for this. This plugin would get substring of artifactId and then set a property. This property could then be used by another plugin downstream in the lifecycle. This option would work, but then I need to create a project for this and maintain it.
   

So, I didn't really like both of these options. Just then I found that, I could use GMaven plugin for this. With GMaven plugin, I could write groovy/java code to get substring. But, then I was wondering how to use that substring downstream in another plugin. After investigating this, found couple of options here:

1. I could use GMaven plugin to write groovy code and then get substring. At this point, I came across maven properties plugin. The properties plugin can read properties file and set them as maven properties. So, I was thinking about writing the substring to a properties file using groovy, which I could then read using properties plugin. Then, I could use this maven property downstream. Even though this would work, it seemed too much work just to get a substring.
2. Then, I found that GMaven plugin is also capable of setting maven properties. Apart from writing groovy code, I could use project.properties.getProperty and project.properties.setProperty to get/set maven properties inside GMaven plugin. This was very handy. So, my solution was just to add this execution to my gmaven plugin:

<execution>

<id>get-artifactid-substr</id>

<phase>initialize</phase>

<goals>

<goal>execute</goal>

</goals>

<configuration>

<source>

String artifactIdSubStr = new String(project.artifactId).substring(10) project.properties.setProperty('myArtifactId', artifactIdSubStr)

</source>

</configuration>

</execution>

Just two lines of groovy code and I was able to use myArtifactId property later on. GMaven plugin came in very handy here.

Using GMaven plugin, we could run an inline groovy script (like what I did above), run external groovy script and also run a remote groovy script. We could also call Java or Groovy classes and call them from here. Having the power of executing groovy and java code inside POM is great. But, we should be careful on how we use it though. If we need to do lot of things, then it makes sense to write a custom maven plugin than to go this way.

Some of the variables that could be used here are project, pom, session, settings, log, ant and fail. If we want to do some custom checks and if the check does not pass we may want to fail the build. In these cases we could implement the check in groovy and then use fail() to fail the build.

You can find more information about gmaven plugin here.

Success Tips for Agile Teams

We've been doing Scrum for few years now. I've also seen how our team and other teams do some of the common things which makes us less efficient. This post just talks about some of the things that I learned through Agile/Scrum training, Agile books, from our own experience and seeing/hearing other teams which do Scrum. Each point starts with what is good to do followed by (in brackets) how teams commonly do it now which makes it less efficient. Many of these were done by our team too, but we are trying to recognize the issues and improve them continuously.

1. Start with product vision and release objectives. Then do road map planning to define when each functionality is needed. It is important to share this with the team so that team understands the big picture and knows where they are heading to. (Some teams go from sprint to sprint heads down, which means the team may not understand the big picture.)
2. When release objectives are set, do release planning with the team and get commitment on what can realistically be delivered for the release. (Management could arrive at the deadline/feature list without consulting the team, which means it is not realistic. Having team commitment early on in the project is very important.)
   
3. Product owner should do continuous backlog grooming and thin slice the user stories with the help from the team. (Avoid having big stories which are not flushed out properly)
   
4. The focus of the sprint should be on maximizing delivering features which end users can use at the end of every sprint, even if it comes at a small cost of developer productivity. (The focus of the sprint should not be to manage the team's tasks and fully load the development team. Instead the focus should be on getting completely implemented user stories.)
5. The team should be well cross trained i.e anyone in the team should be able to take any tasks (as much as possible) to complete the sprint stories. For this to be successful, it is important to have a small team focused on a particular area. (Avoid having teams with too many specialists)
6. Team size including product owners and scrum master should be in single digit. (Think about breaking the team if your team size is in double digits)
   
7. Sprints should be sustainable with each sprint including review, retrospective and planning taking the same amount of time. (Many sprint teams are so tired at the end of long hard working sprint. The focus should be on making the sprints enjoyable and sustainable. Also, avoid having irregular sprint lengths)
   
8. The focus should be on conversation with product owners and recording the conversation as acceptance criteria for the story (confirmation), rather than writing long use case documents. This is based on famous three C's - Cards, Conversation and Confirmation. (Sprint teams should focus on face to face conversation rather than relying on long documents)
   
9. User stories should be accompanied with good acceptance criteria so that team understands what done really means for that user story. (Not doing this is the root cause of many bugs. Having good acceptance criteria and shared understanding (between product owner, developer and tester) of when a story is really done would help a lot)
   
10. Team should do collaborative planning and be empowered to do what it takes to deliver the sprint goals. Entire team should focus on how they can accomplish/commit to the sprint goals together. (Avoid too much focus on individual team member tasks. Instead focus should be on the delivering as a team)
    
11. Estimating user stories in story points is better than doing in hours. (This could be unclear if the team does not have common understanding on what a story point is. Entire team should have common understanding of what a story point is and understand they are relative estimates.)
    
12. When splitting user stories, try to split it vertically rather than horizontally i.e functionally rather than architecturally. (Developers have the natural tendency to break up a big user story into a) DB work b) DB API c) GUI work etc., Instead the user stories should be divided so that each user story delivers some meaningful functionality to the end user.)
    
13. Use daily meetings as a way to communicate to other team members. (Do not use it as a way to report individual progress, which over time might become mundane and not liked by everyone in the team).
    
14. Entire team should be committed to delivering the stories and delivering it completely. (Half/Almost done stories gives a false impression that they are done and also cause more bugs. Avoid it by not giving credit and don't use it's story points to count towards velocity.)
15. When the story is done, make sure it means it is both developed and tested completely (Often teams just do development and call it done, which means testing will be done later. This really means a story is not done, if testing finds major misunderstanding/bugs in the code)
    
16. Team should plan on getting the user stories implemented and give it to testers as early as possible in the sprint. (Often teams might finish a story very late in the sprint, which makes it hard for it be tested and fix bugs (found during testing) before the end of the sprint)
17. Developers could do code reviews, work on infrastructure tasks, help with testing, learn more about the product, etc., if they are done earlier than anticipated. (Often teams pack the sprint which mostly means there is no time to do anything else)
18. Use retrospectives efficiently to find and fix issues which slows down the team. Make progress on these issues transparent so that team can understand that their concerns are handled properly. Continuous improvement is important to any team. (Many teams use it as a formality and not really do anything with the issues raised in retrospectives. This could make retrospective meeting not liked by team members)

I just mentioned some of the things which came to my mind. Feel free to add your comments here.

Books I've read in 2010

I mostly read online articles/blog posts to keep me up to date. I read books whenever I get a chance. Below are some of the books which I've read this year.

1. First, I finished reading the books I was in the middle of reading in 2009 - Programming Groovy and Eclipse - Building Commercial quality plugins. My comments about these books can be found from my last year's post.
2. Java Puzzlers - This is one of the classic java books written by Josh Bloch and Neal Gafter. It has lots of java puzzles which are very interesting and I learned a lot about the intricacies of the language from this book. If you think you know Java very well, try solving the puzzles in this book and it would make you rethink about it. I highly recommend this book to any java programmer and I'm sure everyone (even if you have lot of experience in Java) will learn new things about the language from this book.
   
3. The Mythical Man Month - This is one of the classic books on Software Engineering and Project Management. This book was originally published in 1975 and republished with few more chapters in 1995, but almost all of the things explained in this book still hold good. The classic saying "9 women cannot produce a baby in one month" comes from this book and the author says this to explain that adding more programmers to a delayed project will not make it faster. Some of the ideas presented in this book are explained in the wiki page of this book. I highly recommend this book to any programmer, more so to anyone who manages a project, be it a Development Manager or Project Manager.
4. Design Patterns: Elements of Reusable Object-Oriented Software - I have been using design patterns in my code for several years now and mostly learned design patterns by reading online articles and blog posts. I have read this book on a on-demand basis in the past and always wanted to read it completely. Finally, got chance to read it completely this year and it was a good refresher to many of the design patterns and I also learned few new ones. I don't need to tell about this book, as it is a Classic Design Patterns book and is a must read for any programmer.
5. 97 things Every Programmer Should Know - This book is a collection of thoughts on things which every programmer should know and has many nice ideas and tips. If you are new to programming, I'm sure this book will open your eyes. Even if you are someone with several years of experience, I'm sure your time will still be well spent reading this book and you will learn useful things. You can also get an online version of the contributions appearing in this book.
   
6. Agile Software Development with Scrum - This is a basic book on what Scrum is and explains the concepts very clearly. One of the authors of this book is Ken Schwaber, who is one of the co-founders of Scrum. I have been using Scrum more more than 3 years now and have been ScrumMaster for most of this time. I have read parts of this book few years back and so wanted to read it again and it made some of the things more clear to me. I highly recommend this book to anyone interested in Scrum. If you are thinking about starting to use Scrum, but not clear about it or it's benefits, then this is the book you need to read.
7. Kanban and Scrum - making the most of both - Kanban and Scrum are different Agile project management methodologies. This book explains about Kanban and Scrum - it's similarities and differences and how to use them together to get the advantages of both. You can get an online version of this book from InfoQ. You may have to register/sign-in first before downloading the book. I recommend this book to anyone who is using Scrum or Kanban or any other Agile management methodologies and is a must-read for any ScrumMaster.
8. Agile Retrospectives: Making Good Teams Great - This book is the bible for Agile Retrospectives and is written by Esther Derby, who is well known for her thoughts on improving retrospectives. This book presents many techniques on how to gather data and generate insights in a retrospective. It also presents ideas on how to decide which ideas to implement immediately. This book offers good advice on how to lead retrospectives and is a must read for anyone who leads the retrospectives, mainly for any ScrumMaster. We used some of the ideas presented in this book and they worked great. We also plan to use more techniques presented in this book in upcoming retrospectives.

Looking forward for a very exciting year 2011. Wish everyone a very Happy and Prosperous New Year!

Managing dependencies in Maven

Managing dependencies in Maven, particularly managing the transitive dependencies and versions of the dependencies used can sometimes become a daunting task. Thankfully, there are some maven plugins and other resources to help in this. Just thought of sharing them in this post.

1. Intro to maven dependency management - An intro to maven dependency mechanism can be found here (http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html). This article gives an introduction about transitive dependencies, dependency scopes and dependency management in maven. This is a must read for any maven beginner.
   
2. Maven dependency plugin - You can find the goals and documentation for it here (http://maven.apache.org/plugins/maven-dependency-plugin/). The goals that I use frequently are dependency:resolve (resolves the dependencies and displays the versions used), dependency:tree (resolves the dependencies and displays them in tree structure - an easy way to find out which dependency brings which transitive dependency etc.,), dependency:copy-dependencies (copies all the dependencies including transitive dependencies to a specified location). dependency:analyze could be useful too.
   
3. Maven versions plugin - You can find the goals and documentation for it here (http://mojo.codehaus.org/versions-maven-plugin/). The goals that I use frequently are versions:display-dependency-updates (tells which dependencies use the latest versions and which ones have newer versions available - could be very useful to see how stale your pom is), versions:use-latest-versions (searches pom for dependencies which have newer versions and replace them with latest version - could be useful to bulk update your pom to use latest version of all dependencies).
4. Maven Repository Browser - This is a website (http://www.mvnbrowser.com/index.html), which I just encountered today while trying to search for a missing dependency in my pom. POM report (http://www.mvnbrowser.com/pom-report.html) seems interesting. You can copy paste your POM or just the dependencies section and this would tell you what are the available versions for all dependencies in your POM and if you are using the latest one or not. It also lists what other dependencies are in your classpath. Seems like a simple UI version of what maven dependency and versions plugin does. Apart from this, it also lists the licenses involved. I have used this website very little as I just found it, but it seems useful.

I just mentioned the goals I use most. These plugins have many other useful goals as well, which you might want to explore. Try these maven goals in your maven projects and see for yourself the power of these maven plugins. Hope this post introduced you some new things to manage your maven dependencies and pointed you to some good resources.

Effective Leadership/Communication skills needed for a Manager

Recently, I got promoted to a development manager and being primarily a technical guy before, I'm trying to improve my leadership/communication skills. Below is my personal notes (gathered from various sources) on the qualities to possess to become a great manager.

1. Technical Subject knowledge - Need not be expert in all areas, but need to understand things well enough to make better decisions.
   
2. Be assertive, not aggressive though.
3. Be cool under pressure.
4. Competent.
5. Integrity/Honesty.
6. Ability to delegate tasks.
7. Conduct meetings effectively (proper agenda, starting/ending on time, insisting full participation etc.,).
8. Empathy - Understand and relate to individual's issues.
9. Body language and tone is important (sometimes more than content).
   
10. Tone is very important in meetings, conference calls and even on the email.
11. Smile (even on the phone) helps a lot.
12. Different people in the team want/appreciate different things. Need to understand that and communicate appropriately. There is no one way to communicate to a big diversified team.
13. Create a positive atmosphere for the team and promote collaboration.
    

This list is not comprehensive by any means and just serves as a reminder/personal note to me at this point, so that I can improve in these areas. Just thought to add it as a post, in case it might help someone else. If you have other points, feel free to add comments.

Key Points for any Project (Traditional or Agile)

I'm using Scrum for the past 3+ years and have been ScrumMaster for most of this time. I embrace and advocate Scrum, but last week, I attended a FredPryor Seminar on project management, mainly to better understand the similarities and differences between traditional project management and Scrum. This also helped me in appreciating how basic project management skills applies to any industry. I want to highlight some of the common basic points which came up in the seminar, which are important irrespective of which methodology we are using.

1. Kick-off meeting is important when starting projects (or Sprint in Agile) - I see this as similar to Sprint Planning in Agile. Everyone involved need to get a clear idea on what they are building.
   
2. Team buy-in from the very beginning is critical.
3. Brainstorming with the team on topics is important.
4. Everyone (Team, Management, Stakeholders) agreeing on specific, realistic goals is important.
5. Clear project definition and scope is very important. Avoid scope creep.
6. Be SMART when defining a project (or Sprint in Agile) - Specific, Measurable, Achievable, Realistic, Timely.
7. Risk Assessment - Understand constraints between Time, Cost and People. Often, when risk comes the trade-off is between one of these.
8. Progress - Gantt-Chart in traditional project management becomes Burndown chart in Agile (at least on the high level).
9. Meetings - Need to have team meetings on time, in regular schedule with a specific agenda. Full participation of team is important. Also, need to respect other's time and end the meeting on schedule. Finally, need to take meeting notes and follow up promptly.
10. Team working collaboratively is very important. This will create a co-operative and productive atmosphere.
11. Start the project with self-esteem and end it with self-esteem (for the team).
12. Always recognize the team and celebrate success.
13. Review all outstanding issues after the project (or Sprint in Agile) is done and see what can be improved - Similar to Sprint Retrospective.

I understand the differences between Traditional and Agile and how Agile improves the success of the project. But, still there are many basic things that apply to any project management methodology. Many of these seem like common sense, but if we think about it, lot of these are not being practiced in our day to day projects. I'm sure following these points will surely improve the success rate of any project. I just listed some of the points, which comes to my mind at this time. If you have other points, feel free to add comments.