As a software consultant, normally the need for a solution comes from a client. A client has identified an area where they believe greater efficiency or profit can be generated from a newer software-related solution. They may have seen something that suggested new ideas and ways to accomplish business needs exists "out there." They may have internal people who envision a new set of technology to solve the problem. Or they may simply want to know if anything better exists.

But sometimes, as a consultant, I keep an eye on new trends and ways to plug together newer technologies, and I present these options and opportunities to my customers as often as I can. The solutions that cloud computing can provide. How IoT relates to typical business operations, especially in manufacturing (IIoT). Big Data and data modeling solutions. ETL processes and how to integrate more and more systems. Setting up automated workflow processes. The options and opportunities are vast.

Thomas Friedman wrote a book in 2005 called The World is Flat. No, he is not a "flat-Earther." He was making the case that with modern technology, especially the internet, we are now competing with potentially everyone on the planet. And this has changed technology, especially software. People across the globe are developing solutions for every possible need and desire out there. So as software developers, we have to be keen on determining if a solution in one form or another already exists. To compete, we sometimes need to realize that we won't be writing the solution, but merely implementing it.

The fact that so much software already exists out there has altered the core tasks of a typical software and data engineer. Today, more often that not, software libraries are being "stitched together," leveraging what already exists and is reliable, and creating new solutions from creative integrations. Even Microsoft recommends the use of all sorts of non-Microsoft libraries of code, such as NewtonSoft's json.net JSON library, or the Serilog logging project.

In plenty of cases, though, a current solution does not exist, and this allows the software engineer to spring into action, considering new options, technologies and ways to create an elogent, efficient and reliable new system.

Okay. We have determined that there is not an existing option or set of tools available to solve our current need. As a software engineer, I must admit that I like landing here. New options, new ideas, new solutions. Fun. So how to go about all of that? This is where things get tricky.

We have to answer a whole slew of questions before we can come up with a set of new options. These will include:

• What is the timeline?
• What is the budget?
• What platforms are required?
• What level of security is needed?
• How extensible must the project be?
• Will the project proceed in phases?
• Are there internal politics to overcome?
• What bandwidth will be expected, and can we project future scaling?
• What will the monthly cost be to maintain?
• Will there be internationalization or localization needs?
• What will the support system look like?
• Are there legal matters to review?
• Who will maintain the project long-term?

Once we answer the above key questions (and many more), we may decide that the project is not even worth it, or that there are obstacles that must first be overcome.

Or we may determine that there is, in fact, a good opportunity. Yay. So we go on to researching and brain-storming new options.

At the beginning of the exploration process, it is good to have more than one option (high level). It tells your client that you are trying to exhaust all options, and one of the options may trigger a buy-in quicker than others.

The exploration of new options should be done in an agile fashion. Do some research, perhaps whip up a few demos or Proofs-of-Concept (PoC's), and meet with the client. Some options may simply be dead-ends, others may look promising.

Finally, we arrive at a set of options, or perhaps we have landed on a single, promising solution for the client. We review, discuss the details, and explain the pro's and con's, as well as the risks and potential ROI.

If the solution is a product, we need to go on to the next step-Market Viability. If it is simply an internal solution for a client, then we can proceed to the last step-Realization.

At the step of Market Viability, we go from having a potentially great idea to the questions of how much will this cost to create a Minimum Viable Product(MVP)? How long will it take? And, most importantly, is there a market? That is, will enough people or companies want to buy this, and will the anticipated revenue make a profit ultimately?

One of the challenges of Market Viability is the fact that everyone, everywhere is working. As Charles Phillips, CEO of Infor Global Services, has said repeatedly, "You can never let up in tech, because your competitors are always right there, next to you." So you have to not only look at what your competitors have now, but what they probably will have, by the time you are ready to launch your MVP. Hard questions.

And now at the final step. Exciting. You have determined that you have an idea that can be solved with a solution designed and architected by a group of smart individuals, and you have also decided that it will solve the internal technology gap, or even better, be a solution that can be marketed and sold to other entities.

And so now is the time for real software development planning. How to manage the team(s)? What tools and resources to use to maintain the process, and keep things moving? How to keep the stakeholders informed and in the loop, so you do not go down a bad path and end up with something unplanned for, and unuseful at the end of the day.

Also, how to deal with adversity? Competitors appearing out of nowhere. Development team conflict or disorganization. Unanticipated technology hurdles and limitations. Loss of key team members. The list can be dizzying, so the more a team can plan for such contigencies, the better.

I have seen numerous projects come to fruition, and I have seen catastrophic outcomes. Tech is no easy field, and the way to succeed has to start with the leadership. Any project that has an even moderate size can be compared to a large ship traveling across the ocean. There is a motor propelling it, and someone steering. Weather changes, currents change. Other ships must be avoided. Running aground must never happen. Pirates may be encountered. Your cargo must remain intact. If success, then your shipment of freight makes it to harbor, and a new load is made. Naivete has no place in the world of software solutions.