Recently in Testing Category
The installation process for the Atlassian Starter suite - Crowd, Bamboo, Fisheye, JIRA, Greenhopper and Confluence - is quite daunting and takes about 5 hours+ (way more on my Parallels VM setup, but I did expect that).
It's obvious that the different Atlassian products have been built by different teams, at different times and sometimes even different companies. Although, AFAIK, all products are built with basically the same base technology (J2EE), each product has some minor differences in
- Installation
- Configuration
- Directory set-up
- Starting up / Stopping products (e.g. there's no shutdown command for Crowd, Bamboo automatically installs as a service)
- Configuration files
If the suite has to be installed manually, consistency in the setup process trumps everything. This is even more relevant if the suite is installed by a non-IT, non-Java plain old-fashioned C++ hacker like me.
Generally, editing the configuration files was no big deal, although the sheer number of changes necessary induced cross-eyes at times.
Including Crowd into the installation process made the setup process quite involved and complicated. Although single sign-on is quite a feature, I wouldn't consider it crucial for a 10 user set-up. I would've preferred to make integration with Crowd an optional exercise. Plus, removing Crowd from the standard equation would have enabled more detailed feedback on setting up the different applications's integration features.
Kudos to the Atlassian documentation team responsible for the detailed step-by-step descriptions. It was close to perfect, just very very minor errata in terms of version numbers. A few more screenshots would have been helpful, but would have made the endeavour of documenting the suite's installation process not only daunting, but outright impossible to maintain over time.
I was very disappointed that Crucible was neither part of the exercise nor part of the $10 offer. Atlassian, please make Crucible part of the Dragons exercise and part of the $10 / 10 users offer. I'm sure there were very good technical and/or business reasons not to include it, but if the Atlassian team can pull of a stunt like the Dragons exercise, I know they can pull off including Crucible, too. It just takes a few more beers, I suppose. German beer, of course. :-)
This looks like the start of an extremely interesting series: Quality is Dead #1: The Hypothesis.
Another factor contributing to this crisis are insanely short release cycles (even for desktop software) and the expectation that each major update has to introduce substantial new features. Lot's of them. There's no more time to polish & deepen existing features.
Even if you are not into .NET, WinForms etc. this series by Jeremy D. Miller is a great one to follow. Posts so far:
- Preamble
- The Humble Dialog Box
- Supervising Controller
- Passive View
- Presentation Model
- View to Presenter Communication
- Answering some questions
- What's the Model?
- Assigning Responsibilities in a Model View Presenter Architecture
- Domain Centric Validation with the Notification Pattern
- Unit Testing the UI with NUnitForms
- Event Aggregator
Plus, as a bonus, you will notice that Jeremy is not afraid of using really long function names:
void CloseTheScreenWhenTheScreenIsDirtyAndTheUserDecidesNOTToDiscardTheChanges()
And I'm mentioning this not to make fun of Jeremy.
Erkan Yilmaz tagged me with a few questions regarding testing. However, I'm not a tester, I just happen to be interested in some areas of testing, e.g. unit testing & test-first-coding.
Could you tell something about your first tests?
I started getting interested in unit testing in the late 90s when XP came up. Still not 100% sure about where unit testing ends and acceptance testing begins.
What would you like to highlight as an important thing of testing - from your personal experiences?
As far as unit testing & test-first-coding is concerned, the most important aspect is "Just doin' it even if you think you can't do it".
Why is testing not trivial?
Testing is always hard for a software developer because his/her brain knows about the inner workings of the code and therefore it requires an enormous amount of discipline to thoroughly test the code.
Test-first-coding is hard because it requires to change your mind set from testing as an afterthought to testing before you start coding.
What do you do after testing at work?
Coding :-)
How do you think testing will evolve in the next 13 years?
Most of the kind of testing I'm interested in very much depends on the way the code is structured. So I would guess that the most interesting things will happen round tools & patterns which enable testable code.
Roy Osherove on Object Oriented Testable Programming:
...in many ways, pure object oriented design does not go well hand in hand with the notion of testable design.
...by The Braidy Tester: R E S P E C T.
No, this isn't a subtle hint to anybody in particular. It's just a great post on a very important topic.
Michael Hunter is doing a great job with his blog: THE BOOK OF TESTING - Thoughts From a Braidy Tester.
This is required reading for everybody involved with software testing.
I have setup (and will maintain :-) an RSS feed for all entries in the "You are not done yet" category of this blog.
You Are Not Done Yet: Setup
This blog looks like a worthy addition to the RSS subscription of anyone interested or involved with testing.
J. Timothy Kingblogs about 12 Benefits of Test Driven Development:
The first thing you ask is not “What code will I write?” The first thing you ask is “How will I know that I’ve solved the problem?”.
(Via MemoRanda.)
Oren Ellenbogen started a great series of posts on Test-Driven Development. Well worth following:
(Via ISerializable.)
Keith Ray and Chris Hanson (and others, I bet) have commented on Bruce Eckels post "Programming as Typing".
I would like to take this (wrong - as those fine gentleman have proven) analogy and add a slightly different twist to it - and thus prove that, if viewed from a totally different angle - the analogy works just fine.
(For the sake of simplicity, let's assume that "typing" means "typing & clicking").
If you're not typing, you're not programming - If you're not typing, you're not making any progress. Sounds rather obvious, doesn't it? It's not.
I've seen it over and over (both by observing others and - in rare moments - myself) that a software developer who doesn't type, but sits in front of his computer thinking about the problem at hand for more than 2:30min (I made this number up at this very moment :-) is not actively developing software.
He is stuck.
- Because he's not typing, he is not using all the feedback the code can give him on how the problem should be solved.
- Because he is not typing, he is not actively playing with different approaches to the solution - he is just thinking about the different approaches. He's making up a mental model of the code - and more often than not, this mental model is rather different from what the code does (This happens to the best of us, doesn't it?).
- Because he's not typing, he is not using the feedback which comes from writing & running a simple test. He is not trying to solve a small chunk of the problem to get a deeper understanding of the larger problem.
Typing is more closely related to software development than you think.
(There may be others who frantically scribble boxes on a sheet of paper for more than 2:30min - those are stuck too, not as stuck as those just mediating, but still stuck)
(Of course, this doesn't apply to you if your belonging to the elite group of geniuses - but that's only about 0.0001% of the software developers out there)
"Sure, test automation is a good thing. But we can't, and shouldn't, automate them all. Why then, ask people to 'automate all tests'?"
"The only time not to automate a test ought to be if we are never going to want to run it again. But it isn't. The more common time we don't automate a test is because it would be 'too hard' to automate it. That is, however, not really a property of the test, though we like to put it that way. It's really a property of our own test automation capability and tools."
Hallmarks Of A Great Tester - Great presentation by The Braidy Tester.
I also like the subtitle of his blog: Making developers cry since 1995.
Now, let me state something unequivocally: 98% of the time when you think you've found a bug that is not your fault, it really is your fault. The other 2% of the time... well, it's probably your fault as well.Amen to that.
It's worth pointing out that beta testing is a great way to start a career in software. It looks good on a resume, and it gets you noticed by engineers in your field -- if you submit a bunch of bugs to a company, it shows you have a keen eye and understand software, and they are going to remember your name later. Seriously, we've always known our top beta-testers by name, and they get much respect; someone might say, 'Oh, O'Donnel reported a cosmetic bug in the main panel,' and the engineers would be all, 'O'Donnel? Dang, I better hop on that!'An interesting twist on the topic of dealing with beta-testers....
And, in turn, lots of the guys I've hired over the years first came to our attention as beta testers.
Sam Gentile provides a great write-up on the Test-Driven-Development (TDD) process.
Here's a site which features great bug reports (mostly on topics related to MacOS X / MS Office X). These bug reports are in depth, with lots of screen-shots, suggestions on how things could be improved etc. In fact, I would be glad if the bug reports we receive would be half as good as these posted on betalogue.com.
However (and you knew this would come, don't you), there's one major caveat. The author sometimes uses judgmental language to comment on these bugs and the software developers responsible for these bugs.
Don't do that. Don't use judgmental or even insulting language.
Refrain from using phrases like "I wonder what these guys have been thinking...." or "It's obvious that XYZ doesn't think about..." or "Again, XYZ proves that they don't pay enough attention to....". Don't speculate about issues with the developers code.
State what's wrong. State what you expect. State how this bug affects you. Provide sample documents/screen shots - anything which may help to resolve the bug.
If you're really pissed, still use neutral language. If you're not pissed, but slightly annoyed, use upbeat language. You will be surprised how much this will help a constructive discussion about the bug and how much more attention is paid to your report.
...of TDD.
Michael Tsai responds to Will Shipley's now infamous Unit testing is teh suck, Urr.
Here's one important aspect of TDD Michaels well-balanced post is not elaborating enough:
Unit-Testing is about design (& specification of behaviour) as much as about testing, as many proponents of TDD point out (check Dave Astels blog who's taking this aspect of TDD to the extreme).
While I'm at it, Michael states:
If you need a new class or method, I think it’s a waste of time to write a test that simply checks for the existence of the class or method, then write a stub, then run the tests, and then then flip back and write the real tests.
I beg to differ. If you intend to write a unit test anyway, there's no time wasted doing it with tiny little steps like mentioned above - in fact, I would argue to use the tiniest steps possible, asking the compiler/linker/runtime for help as much as possible). Plus, it saves precious spare brain cycles for those really hard problems. Of course, you're smarter than me, so you probably have brain cycles to spare :-)
Sara Ford has 5 Tips for surviving as a Tester.
Dave Astels proposes Behaviour-Driven-Development instead of Test-Driven-Development.
The I18n G.A.L. : Weblog - Interesting stuff on internationalization & localization.
Exploratory Testing using Personas:
Lesson learned: Get the customer data when they are using the software in their own work environment.
Good read.
A Unit test should test only one thing - if you're doing TDD.
Black Box Software Testing: A course by Cem Kaner & James Bach.
(Via MemoRanda.)
Dale Emery: What Do You Want From Tests?.
The Developer Testing Paradox:
The root cause of most software project failures, and of the poor general health of most software, is the lack of early-stage unit testing.
The metrics presented are a little bit off IMHO, especially when dealing with legacy systems. Still an excellent read.
Two excellent posts on TDD (Test-Driven-Development) by Keith Ray:
Roy Miller on Acceptance testing. Pretty java-centric, but still useful stuff.
Video webcast of Kent Beck's presentation on Developer testing. There's also a 1h mp3 at IT conversations.
Required listening.
