Putting a Price Tag on Bits and Bytes

Software is a tough business, but pricing software is an even tougher business. As obsessive full-time developers we often want as many people as possible to be using our projects, but we also have to strike a balance with the fact there are bills to pay and families to feed.

The price you’re paying for our software has to factor in the costs of ongoing development and support. In our case, the Cerb4 licensing cost is basically just passing around a (slightly-coerced) tip jar for people who like what they see so far so development can continue. As developers we like to constantly be stuck thinking about the future and what new things are possible from the pieces we’ve just built.

Customers tend to approach things differently. Very few prospective users are genuinely thinking about future potential when they see a price tag on software. They take a look through all the corners of a demo and website to get a feel for what is currently possible with the software – probably because they’ve learned over the years that developers can be incredibly undependable in their estimates about new development (and they’re right; but hey, we’re working with ethereal “thought stuff” here).

You’re probably thinking this lead-in sounds like a typical justification for raising prices, right? Well here’s the curve ball… we think we’ve been pricing Cerb4 too high based on its exciting future potential that (for the most part) only we can see right now.

A few months ago we switched over to per-user pricing because it seemed like a great way to offer a lower entry price for companies with few users, while companies with more users could contribute more (based on the theory software seats scale like office chairs). But that’s proven pretty artificial, and for these few months since that decision we’ve been dealing with several tough questions about per-user pricing, like: What if I want to deactivate workers who have left the company without deleting their history? What if we’re a small company that has dozens of volunteer helpdesk workers?

It has recently dawned on us that we’re just setting up an artificial obstacle for people with these worker limits. If you buy into our project, we want you to feel like you own the software. It’s not our style to litter the project with hooks and throttles just for the sake of cash flow. Just like overly-aggressive license validation procedures, it ends up punishing all the paying users out of the paranoia of going broke. The reality is that if we’re doing something genuinely useful we’ll find a way to keep the rent paid, the lights on and the fridge stocked. We may not become the next Microsoft or Salesforce.com with that mindset, but at least we won’t be holding you guys back when you decide to spend a few hundred dollars to use Cerb4 for your mission critical e-mail.

So here’s what we’re going to do: We’re going to remove the per-worker limitations for everyone. Those of you with 5 worker licenses are likely going “Woo hoo!” while those with 25+ worker licenses may be going “What the hell, Jeff?”. Please keep in mind what we’re trying to do here is for the project and community before ourselves, and the fact we may come off as manic-depressive in our search for ideal pricing from time to time has to do with the fact we’re developers, not prescient economists. We’re not fickle, we’re just incrementalists. Each tweak is based on a lot of ongoing observation about the impact of each decision in the real world (which is pretty hard to predict). We’ll continue to find ways to reward people who’s early contributions have helped us grow over the years.

The Impact on Owned Licenses

We’re setting the “owned license” base price for the project to $499 with no artificial limitations. We’ve brought back the small business discount for companies making less than $250,000 USD gross revenues per year which knocks 20% off the price (to $399). Educational institutions get a similar 20% discount (also to $399). Users upgrading from Cerberus Helpdesk 3.x get a 30% discount (to $349, the discount used to be 50% but it was against $995 for unlimited, so you’re still saving about $150 more from this tweak).

The Impact on “As-a-Service” Hosted Licenses

On the hosting “software as a service” end, we’ve stopped tiering pricing by users as well. The scaling issues we face in managing servers have very little correlation to total users, and more to do with how companies manage their e-mail behaviorally (e.g. never deleting spam, receiving lots of work-in-progress attachments). We’ve moved to a much more straightforward pricing model on hosting of just basing it on storage. All hosting plans from today will start with 5GB of storage for $49/mo. 5GB additional storage is +$25/mo and 10GB additional storage (prepaid) is +$42/mo. If this was purely dollars for hosted gigabytes it would be a bit expensive, but this covers the software, upgrades, support, phone support, backups, server monitoring at 4AM, and all those things which you’re glad we’re doing so you don’t have to.

With that Out of the Way…

I can’t guarantee that several months from now I won’t be sitting here telling you something else based on future information and observations; but with all the information and history we have at our disposal right now we feel this is the best direction for funding Cerberus Helpdesk’s ongoing development (so we can continue to reach for all that exciting potential we’re always talking about).

If you have any questions about what we’re doing, how we’re doing it, or what we’re aiming for, ask away! If you’d like to rant, send me the URL to your forum thread and I’ll read through it and post my thoughts without getting defensive or cranky. Nothing is off limits.

Thanks!

-Jeff

Alright! We’ve just done another full round of QA tests against Build 593 to confidently declare it stable. We’ve been using it live on our own helpdesk for about a week.

There are a couple dozens tweaks and improvements included, but our major focus was on bringing knowledgebase management back to the worker GUI, integrating the public knowledgebase with the Support Center (to remove the need for two public tools) and performance improvements.

We’ve also switched over to requiring MySQL 4.1 to vastly simplify fulltext indexes for quicker searches, to get subquery and transaction support, and to move forward with internationalization. It was really useful to gather community feedback about this change ahead of time in the voting booth on the forums.

When we had to shuffle some of our hosted sites around during the server emergency last week, we ended up temporarily putting more sites on a single server than we normally would have liked. It was interesting to observe that about 100 concurrent copies of Cerberus Helpdesk 4.0 ran very usably on a mid-range Debian box (Dual Xeon 3GHz, 2GB RAM, 2×500GB RAID-1). In practice, for that kind of load, we’d throw more memory at the machine and probably bump up the RAID concurrency. The hard disks are almost always the bottleneck if you can’t keep databases entirely in memory.

Naturally we’re not in the habit of overselling a machine’s resources, but it gave us the opportunity to look at bottlenecks with that many copies of Cerb4 running. We used this information to add some last minute performance improvements to Build 593. One of the most effective was improved database indexing on the message_header table – which is used to thread incoming messages with the messages they’re replying to, as well as to display the original headers while reading ticket messages. There were a couple other missing indexes, and a few places in the code we could reduce unnecessary queries.

For the knowledgebase, you’re now able to use nested categories again (as was heavily requested). The top-level categories of the knowledgebase are now called “topics”, and you use topics to decide what branches of the knowledgebase to display on your various public community tool instances. You can display more than one topic for a particular website. We’ve found it’s most effective to use your products and services as the basis of topics.

To take advantage of the new community tool changes you’ll need to go into ‘Helpdesk Setup->Community Tools’ and choose your knowledgebase topics to display for each tool.

You can find the full changelog for Build 593 in the forums.

Around 1:30PM Pacific on Monday (April 28th 2008) we had one of our Cerb4 Hosted machines start writing garbage data all over its RAID drives. This corruption turned MySQL database files into directories from random fragments of the disk (website files, etc). It was a complete mess.

We lost SSH access to the box around that time and initiated a remote reboot. When the machine came back up the primary partition was mounted read-only (which pretty much always means no good).

A full file system check is needed, but we decided to take the proactive step of moving all hosted sites from that machine to a standby server rather than letting the corruption get worse.

Our off-site backups didn’t contain corruption, though a few of the on-server backups did. We do our weekly off-site backups early on Sunday mornings (when the traffic is lowest). In the worst cases, we had full databases for any site that’s only missing Monday morning.

As soon as we realized what was going on, we shut down the scheduled tasks on that machine so we wouldn’t be losing your e-mail to a corrupt environment. This buffered up mail in your POP3 accounts. This is our standard procedure, which gives you the ability to access your critical e-mail directly from your POP3 mailbox (with a “leave a copy of mail on server” client).

The way Cerb4 works, the corrupt database tables also caused the helpdesk parsers to lock (as designed) and started buffering up downloaded e-mail to the disk. This, combined with turning off scheduled tasks, should have prevented mail loss during the day.

We took a little longer than we would have liked in restoring some helpdesks this afternoon because we wanted to maximize the amount of recent data we were recovering, instead of just falling back to the backups when it wasn’t always necessary (and would lose recoverable data). For some sites, the only corruption was on an infrequently changed table like ‘worker’ which we could restore independently. The majority of sites on that machine had no corruption at all (and the busiest sites were affected the most).

We currently have all sites and databases restored on a fresh server. We’re importing the /storage directories which contain attachments and unparsed e-mail. These imports won’t affect your ability to use your helpdesk, but you may not be able to open some older attachments until the process is done. We’re establishing another standby server.

After things settle down a bit, we’ll reflect on this 12 hour marathon and think about what we can do to get sites back in action quicker from a total machine failure. The off-site backups will continue to play a big role. Cerb4 is “componentized” enough that our hosted sites are incredibly portable on our network; but we also need to have our standby machines already mirroring the DNS for a seamless changeover.

If you’re still having trouble accessing your hosted site it’s likely just a DNS caching issue from the server migration. Make sure your *.cerb4.com site is resolving to the IP “72.232.216.42”.

Thanks to everyone for your patience during your calls, e-mails and livehelp chats today. This isn’t a situation we’re going to have to go through very often – in fact, it was a complete nightmare — but it’s at least slightly comforting to know our off-site backup process gave us day old data as our worst case option.

[http://www.cerb4.com/forums/showthread.php?t=874]

[Alternate URL: http://forum.cerb4.com/showthread.php?t=874]

Since 4.0 development has been moving so quickly, we’ve just had everyone pulling the absolute latest source code from Subversion (technically called the “HEAD” revision). This was a good thing during early development because it kept us all on the same page.

However, with 4.0 no longer in beta, we really need to improve this process so people know they’re running the latest stable code and not something we worked on 5 minutes ago, or bleary-eyed at 3AM. Most of the time we’re pretty thorough with our local tests, but they’re no substitute for the acceptance tests and browser-compliance tests Joe@WGM runs in Q/A. You guys have access to our main repository, so there’s no guarantee the absolute latest code has gone through Q/A.

Our philosophy with 4.0 has been to *not* make a big deal about point releases in an attempt to discourage ourselves from batching up functionality and fixes to make a “marketable release”. We want to give you real-time access to the fixes and features that you care about. Those of you who need a specific feature (or want to help test) can upgrade 5 minutes after we commit changes. Everyone else can be confident we’ve fully tested the incremental upgrade they’re going through.

It’s not that we haven’t been testing things carefully all along, but we haven’t been very good about telling you after we run all our tests.

So here’s what we’re going to do:

• Every other Monday we’re going to make a new milestone build that will go through Joe@WGM’s entire acceptance and browser test suite. Anything that fails will go back to development as a top priority. This is pretty much how things work now anyway, but it leads into:
• Once the acceptance tests all pass, we’ll update the latest “stable” build on our on-line demos, instant evaluations, hosted helpdesks and the downloadable ZIP file.
• If you run Cerb4 on your own server, you can upgrade by simply issuing the command “svn -r nnn update” from your /cerb4 directory at the server console (where ‘nnn’ is the version number posted on the site). In Windows you can use TortoiseSVN to “Update to Revision” by right clicking the cerb4 folder in Explorer.

Thanks for being enthusiastic enough about the project that this is even an issue!

First off, thanks for supporting Cerberus Helpdesk development by previously purchasing a version 3.0 license! Contributions like yours have allowed us to push forward full-time for over six years.

Over those six years, the project has gone through some big changes while striving to stay agile for a growing community. The early community was mostly made up of techies; but over the past few years we’ve had increasing interest from people in all industries who are still taking the plunge to web-based e-mail from POP3/IMAP mailboxes and desktop e-mail clients.

We’ve never been the type of project to increase major versions for marketing reasons. We increment our major version when we feel the project and its concepts are going through a fundamental metamorphosis that, despite increasing version numbers, doesn’t require us to stay trapped by a compatibility with the inefficiencies and missteps of the past.

The latest metamorphosis has been version 4.0, which is our answer to the question “What would you do differently if you could start over with what you know today?”.

At first glance you’ll probably notice things are different (and slimmer). Please don’t let this deter you, as it’s part of the process. 4.0 places its major emphasis on customization, which means plug-ins can add or change functionality based on the needs of specific groups. It means you aren’t boxed in by a “one-size-fits-all” mentality. This clean slate allows you to add, build or share the pieces you need without having to deal with the bloat of the pieces you don’t.

For example, at WebGroup Media we’ve added custom tools to our helpdesk to manage our forums, CRM opportunities and product licensing. We’re able to quickly know more about our specific customers when looking at a ticket without cluttering the project for everyone else. A big part of 4.0 development is going to be building and fostering the development of similar tools for various groups. For this reason, we want to keep the core (shared) project as slim as possible.

If you feel your past investment in our project has been worthwhile, please consider upgrading to 4.0 and supporting this effort. As a 3.0 license owner you’ll receive a 50% discount. Don’t wait around for the perfect project to suddenly materialize without your input, stop by the forums and tell us what your ideal helpdesk should do.

We’ve been asking you guys what you feel about the new $50 per worker pricing ($25 per worker on upgrades) since we made the switch. The change to per worker pricing has freed us from having to hold functionality hostage, since we really feel every feature is helpful to organizations of all sizes (heck, I use most of them on my own webmail!) So far the reception has been positive, but with a few lingering issues to work out.

The main remaining contention is about where the cutoff should be for unlimited workers. Many people feel that the cutoff being 50 workers is too high. It’s true that the average number of workers on licenses since we’ve been asking has been quite a bit less than 50. We can also sympathize with organizations that have a lot of community volunteer or part-time workers.

Looking at the data from recent orders, we feel a better cutoff for an unlimited license is probably around 20 workers. That still gives a good discount to smaller teams while keeping the price reasonable for larger teams. Unlimited workers after 20, coupled with a small business or upgrade discount, reduces the unlimited worker license by up to 60%.

We’ve decided to trial offering the unlimited license at 20 workers through April 15^th 2008. The 50% discount for 2.0 and 3.0 license upgrades still applies as well. That’s $995 for unlimited workers on new purchases or $497 for an upgrade from 3.0. Both prices include all version 4.0 updates.

If you already purchased or upgraded to 4.0 and paid for more than 20 workers, contact us and we’ll upgrade you to an unlimited license at no cost. If you paid for less than 20 workers, we’d be happy to upgrade you to an unlimited license using your initial purchase price as a discount.

Thanks for the constant stream of candid feedback!

Here’s a quick progress report on the top three responses (by a huge margin) from the “Which 4.0 major feature do you want WGM to focus on next?” poll in the community voting booth.

• Time Tracking: I just finished a big Configuration area cleanup to allow plugins to contribute new tabs. Time Tracking needed its own little setup area. Beyond that, this is a really straight forward plugin/feature. In Devblocks-speak, we just need to create the DAO/Model/AbstractView for time slips and allow them to link to Organizations, Tickets or Calls like Tasks work.
• Support Center Revamp: The “Communities” area was finally moved into Configuration area since it’s an administrator feature. The Configuration area cleanup also made Community Tools a lot easier to configure and install than they were on the old communities page. The next steps here will be combining the Knowledgebase (KB) and Support Center (SC) plugins to reduce setup confusion, and to allow the SC to show KB articles. We’ll likely bring back KB categories, but unlike Cerb3 you’ll be able to have multiple KBs. Workers will be able to manage the KB from the worker GUI instead of having to create and use editor accounts on the KB community tools. We’ll also finish up the documentation for making new SC themes.
• Translations: We finally sorted out the Western Latin special character issues that have been plaguing Cerb4. This was a big step to getting translations going, since externalizing the strings would have been pointless if characters like umlauts wouldn’t display properly. Now we just need to move the rest of the strings (i.e. words, sentences) out of the templates and into the strings.xml files in each plugin.I’ve been thinking about creating a simple translation plugin that translators can use to edit the strings.xml files from the GUI with side-by-side comparison of English and the destination language. We’re using the TMX standard for localization, and Cerb can take care of formatting everything properly without translators having to edit XML across a dozen different plugins. A built-in TMX editor could also give translators a list of only new strings, which would be a big time saver versus scouring files.

It’s not too late to add your feedback to the poll. Stop by the Community Voting Booth in the forums.

We started Cerb4 with the idea of supporting MySQL, PostgreSQL, MSSQL, Oracle, etc. As development has progressed, we’ve hit several issues where our database functionality needs won’t abstract or overlap effectively.

The major benefits of Cerb4 standardizing on MySQL 4.1+:

• Fulltext searching: Right now we’re implementing database-driven functionality where our supported platforms (MySQL and PostgreSQL, at the moment) overlap. On text searching, this currently centers on basic SQL-92 “LIKE” syntax searches. The downsides here are pretty obvious; there’s no boolean support, wildcard searching is grossly inefficient when looking for multiple words and the order of keywords must be precise. The ideal situation is to have the full feature set of MySQL’s FULLTEXT index type, which, while not perfect, opens up a more Google-like syntax that’s much more useful than what we’re currently doing.
• Internationalization: It’s much faster and cleaner to be able to depend on MySQL’s encodings and multibyte behavior than to abstract how each platform approaches i18n. MySQL has its own tricks too, like “SET NAMES utf-8;” being issued per connection to ensure the client and server are handling multibyte encoding the same way.
• Sub-queries/Foreign Keys/Cascades/Joins: With day-to-day functionality we haven’t had much of a problem avoiding sub-queries or foreign keys for MySQL 3.23’s benefit. However, to support this rather old version of MySQL we lose the ability to be efficient in database patches and maintenance tasks. While not exclusive benefits of MySQL 4.1+, cascaded deletes on foreign keys, sub-queries and joins in UPDATE/DELETE would make patching and maintenance much more efficient. We end up with a lot of N+1 queries presently, where if ‘N’ is based on a table like ‘ticket’ it may end up around 1 million repeats of a query. This ends up clashing with the default PHP execution time of 30 seconds (which is fine for rendering a typical page, and an eternal pain for upgrading/maintenance tasks in PHP when the execution time can’t be overloaded at script level).
• Transactions: Another needless concession to MySQL 3.23, without transactions we have a much harder time making sure partial data isn’t written to the database — especially while parsing tickets. Transactions ensure everything is successful or none of the changes are applied. We’ve always needed this, and MySQL 3.23 support has always been in the way.

All of these issues contribute to the speed of development.

That’s the case to be made on our end. Making this change would unstick several big feature requests, that are far more important to us than the marketing bulletin of ‘database independence’. We’d really like to hear from you if this change would be a problem for you.

Please take a moment to vote on this issue in the forums.

I think we’ve solved most of the issues with special characters (such as umlauts) in Western, single-byte languages for Cerb4.

Cerb4 Build: 563

Due to a few lingering obstacles for full internationalization, we’re still using ISO-8859-1 encoding. That should completely cover English, German, Spanish, Italian, Norwegian, Icelandic, Swedish and others. It should mostly cover Dutch and French (with the exception of a few rare characters).

Warning! Dear Non-Techie friends, the rest of this post gets a bit nerdy.

The recent issues were caused by a couple things:

• META/Content-Type: We were a bit over-eager and had the browser using Unicode (UTF-8) encoding while the data was still Western Latin (ISO-8859-1). Display-wise this is entirely fine because of backwards compatibility — but when input, like forms, was sent from the browser to PHP, it could include multi-byte characters that the database would split up into seemingly random single byte characters. This affected things like umlauts and the British pound sign, not just fully multi-byte languages like Japanese, Chinese and Russian.
• Ajax/XHR: Modern browsers default their XHR requests (how Ajax happens) to UTF-8 encoding when no Content-Type header is provided. This is why even with ISO-8859-1 encoding, umlauts would break on dynamic functionality in the helpdesk (ticket peek, reply, templates).
• UTF-8 vs Unicode: Unicode is a variable byte encoding which overlaps with ISO-8859-1 completely on the first byte. UTF-8 overlaps with ISO-8859-1 over the first 7 bits (which is basically all the characters printed on the keys of a standard US Keyboard, including the SHIFT characters). Extended characters, such as umlauts, are expected in a second UTF-8 byte (0xC3BC vs 0xFC).

We’d love to switch over to UTF-8 immediately and support almost every language, but here’s what’s in the way:

• PHP6 is on the way with native Unicode support for all strings. This is much cleaner than anything we can do with the mbstring extension.
• mbstring function overloading (e.g. substr() -> mb_substr()) can simulate native UTF-8 strings with PHP5, but it requires php.ini changes. These changes will usually affect all PHP scripts on your webserver. Under ideal conditions we can control this with Cerb4 and only affect our script. However, real-world conditions are rarely ideal and this would make for a terrible minimum requirement.
• Our database abstraction (MySQL, PostgreSQL, etc.) makes UTF-8 support in the database layer more complicated. This would be much easier if we knew we were always dealing with MySQL 4.0+. We may need to make some tough decisions here (though they’d allow us a lot more efficiency on things like fulltext index searching versus our current watered-down alternatives).
• Converting existing databases to UTF-8 isn’t a one-click process. Ideally we’ll only have to help people convert their existing database if they need UTF-8 support. It can also be the new default. Our current goal is to make sure we can support UTF-8 optionally without breaking every existing Cerb4 helpdesk to get it. Data-wise we’re probably fine here, but we’d run into the same issues as described above until the database was converted to UTF-8. The MySQL default is ISO-8859-1 (which it dubs “latin1”).

UTF-8 hangups aside, this should fix the main issues with Cerberus Helpdesk for our friends in Germany, Austria, France, Belgium, Italy, Sweden, Norway, Spain and the Netherlands (who have been rightfully banging down our door about it!)

After many months of hard work, we released Cerb4 into a world of overflowing inboxes and overworked people. A crowd started to take notice.

“Welcome to the party!” we said.

“Oh, this is a nice little get-together,” said the crowd. “But it’s not a party without Custom Fields!”

Being good hosts, we promptly set down our plastic cups of punch and disappeared into the WGM development labs. Keyboards began to click away rhythmically like locust. The shouts of developers rang out, intermingled with the squeaking of whiteboard markers.

Some time passed. Suddenly, the doors of development burst open and we programmers emerged, looking like zombies; but smiling — unlike any zombie I’ve ever met.

“Everybody,” we shouted in unison, “we’d like to introduce Custom Fields!”

Then we had ourselves a party.

(That’s pretty close to what happened, anyway.)

Late-night delirium aside, here’s the scoop:

• You can now create custom fields at the global and group levels. At the global level you may want to re-create custom statuses or ticket priorities. At the group level you can track more localized data, like product or department-related fields.
• You can view custom fields as columns on any worklist.
• You can use custom fields as search criteria.
• You can associate custom fields with the situational questionnaires on the Support Center to have answers captured in ticket fields automatically. This also allows you to build more detailed contact forms, using single-line text fields, multi-line text boxes, dropdown menus, checkboxes and date fields.

Enjoy!