Common Optimization Misconceptions

When we talk about optimization with potential users and customers, we usually get pushback on a few common points, which we will discuss in detail here.  If you’re confused about what we mean by ‘optimization’ (which is common, because it’s overused), you should probably read this first.

Common Misconceptions About Optimization

1. My operation is too complex to capture with software.

This is probably the most frequent misconception we hear related to optimization or automation software.  

While it’s a genuine concern - logistics operations are rarely simple - it’s certainly not impossible to solve.

The easiest way to think about it is probably in the context of a decision tree - the decision tree for a given operation may be extremely complex, but you can almost always express all the decisions that a scheduler would consider in such a tree.

This type of decision tree, when combined with the rules governing an operation, can represent almost any type of scheduling or routing optimization problem.  

Typically, the hierarchy for the order in which rules are applied looks something like:

  • Legal operating rules (duty hours, minimum rest, etc.)

  • Organization operating rules (quality-of-life policies, guaranteed hours, etc.)

  • Flexible, but preferable, rules (quality-of-life preferences, seniority preference, equal training opportunities, etc.)

The decision tree really becomes important in this third step, where most schedulers or operations managers feel that their decisions can’t be represented.  In reality, it’s usually just a matter of figuring out the edge cases, and adding these as conditional rules.

Obviously this process means that the software must be configured for each particular operation, but a well-built flexible platform should make this possible.

2. Our operation does things on-demand, so there’s no possibility for optimization.

The first question we usually follow up with: is it really on demand?

Many operations operate “on-demand”, in that they allow bookings up until a given time.

In most cases, however, that time window isn’t really the moment before a task is operated - there are a few exceptions, which we will talk about in a minute - but generally, some portion of the bookings are pre-scheduled.  That allows time for schedule optimization.

In such a situation, the optimization might work like this: you receive 60% of your bookings at least 24 hours in advance, run an optimization, and develop a base schedule.

As the other 40% of your bookings come in during the final 24 hours, you can run a series of re-optimizations, but since the original problem has already been solved, this will take much less time.  You can do this on an ongoing basis (rolling time window), making it much easier to stay on top of things, rather than locking bookings at some point, or dispatching as you go, which sacrifices efficiency.  The other benefit of this method is it allows you to take into account things like cancellations on the day of operations, if that’s relevant for your particular operation, further increasing efficiency.

Sometimes, however, operations truly are on-demand - you can think about bike-sharing, taxis, etc.

In each case, there are still possibilities for optimization.  In all cases, you can examine historical data, extract trends, and then make decisions about how to reposition to improve efficiency, whether that’s better meeting demand (bike sharing) or reducing the cost of empty distance traveled (taxis).  

Taxis lining up at the airport is a crude example of this - they know this is a centre for demand.

Once a model for this is developed, you can integrate that model with on-the-go scheduling optimization to improve efficiency even further compared to a dispatch model.

3. Our schedule changes all the time so optimization wouldn’t be possible.

This is a sub-case of 2., and our typical first question is much the same - does your schedule really change all the time?  Or does a portion of it change?  And if so, what proportion changes?

The solution is often the same as the above as well - knowing, for example, that 40% of a schedule is going to change, we can create our best-guess schedule, and then re-optimize the 40% that changes throughout the day, as changes come in.  

The other opportunity with a changing schedule is to apply machine learning and other techniques to predict changes.

Ever wonder how Amazon can guarantee 1-hr delivery in major cities?  They can’t keep enough stock of every item for every person in the city; they can, however, predict that on Monday, 400 people are going to order a particular item, and how that varies daily, weekly, seasonally, etc., so they can stock appropriately.

The same can be applied to operations - we might not know which exact trips are going to be canceled, but with enough data, it’s usually possible to predict approximate volume and location to create schedules that are more robust when cancellations occur.

4. Our fleet is all the same, as are the qualifications of the crew, so there’s really no opportunity for optimization.

Let’s take the example of a business jet charter operator that only operates one type of plane, and therefore only has pilots that are qualified on those planes, and operates a floating fleet.  Essentially the Southwest Airlines model, except flights aren’t flown on a pre-set schedule, but instead booked according to passenger preference.

You might not think there’s an opportunity for optimization - wouldn’t you just deploy the closest plane to a passenger request and be done?  Well yes...and no.

Sometimes that is the best option.  But there are other factors to consider - while the planes are floating (not fixed to one base), there are variable costs depending where they’re being kept.  The crew themselves live somewhere, and generally like to be home when they can.  Some routes are more popular than others, and as a result are easier to sell (less likely to be empty legs).  For long stays, you need to fly the crew home on commercial, and send out another crew.  Planes need maintenance at different times.

The sheer number of combinations that are possible makes it impossible for a human to explore, even when the number of variables is reduced by keeping a uniform fleet and crew.  As a result, it’s almost always possible to improve efficiency when using optimization, and at the very least, you can gradually increase the complexity of the rule set and variables as you get new ideas, or factors change, while with humans you’ll almost always reach a saturation point very early (we’re sending the nearest plane).

5. We’re dealing with a shortage of crew, which is more important right now than optimization.

The two are actually closely related.  There’s a shortage of crew (pilots, drivers, etc.) in several industries right now, and we’ve heard the concerns of those leading various operations.

In a shortage, the crew have many opportunities - they can go to another operation that offers better training or advancement opportunities, higher pay, better quality-of-life, etc.

Employee turnover is disruptive for any firm, and we’ve talked before about how we believe the cost impact is undervalued, but ultimately, if you can’t recruit enough crew to run your operation, you’re going to suffer lost revenue.

Typically the answer to crew shortage is being able to offer the best perks for crew - higher pay, better benefits, better quality-of-life, etc.  Sensing a trend yet?

These are all things that can be improved with optimization - reducing costs will give you higher margins to spend on crew.  Being able to build schedules that work and provide a guaranteed number of days off, or a guaranteed number of nights at home, is all part of what optimization can help provide.

Ultimately it comes down to having control of your scheduling process, knowing how changes affect the crew and your operations, and trying to be the most efficient operation you can be, so you can spend on other things.

6. Optimization is too expensive for my company.

This used to be more true than it is now, in general; historically, most high-quality optimization was offered by niche consulting firms, and while they often provided great results, their cost structure and incentives weren’t in alignment with their clients.

Thankfully, with the relatively recent move towards cloud-based services and APIs, incentives have changed, and there are many options for those considering optimization.

Make sure when you’re looking at options to try and find someone with expertise to really evaluate the technology.  There should be an affordable option for you.

The other thing to consider when evaluating the cost of optimization technology is that it is an investment (or should be) - you should be able to reap future benefits that will easily outweigh the upfront cost, and if you’re not convinced, ask the provider to justify it for you.  They should know how you will benefit and be able to make the case as to why you should invest.

7. I tried an optimization company and the results sucked - I’ll never go back.

This one is tough.  We feel for you if you’ve ever tried optimization and it didn’t work.  We’ve all dealt with consultants and providers that didn’t meet expectations or didn’t produce what they promised.

The tricky part with “optimization” is that the word is ambiguous; we use it when referring to the particular mathematical optimization technique we use, and the situations in which it applies, but it can be used in a lot of different contexts and mean many different things.  We wrote a bit about that here.

Generally, we believe disappointment usually comes as a result of two things:

  • Optimization technique limitations: the technique that was being used wasn’t powerful enough to handle your problem in a reasonable amount of time.

  • Flexibility: the tool you used wasn’t flexible enough to generate results that were useful to you (and this is often a result of the mathematical technique used as well).

The good news is that the abilities of mathematical optimization providers use widely different techniques, so the poor experience you had isn’t representative of all providers.  It’s likely there’s someone out there who can provide awesome results for you.

The tricky part is that it’s often difficult to distinguish who that provider is unless you have a background in the technology.

Our main advice would be the following:

  • Try and find a third party that knows enough about the mathematical background to interview the provider and evaluate the technology.

  • Ask the provider about previous use-cases, try and find one that directly mirrors your own use-case, and get into the details of how that works, and how it went.

  • Ask for customer references (when far enough through the process).

Our hope is that you won’t give up on finding some great optimization, as we know how dramatic the results can be once you do!

Hopefully with these in mind, you have a better idea of what optimization can provide.  What other preconceptions do you have about optimization?