Effectiveness vs. Efficiency

My impression is that over the last years the two words effectiveness  and efficiency are used more and more interchangeable. This is very misleading due to the totally different original meaning. To explain this, we can take both words and use it in a business context.

Effectiveness means that my business generates the wanted effect. My business idea is implemented and working. If I am a baker, it would mean I am able to bake bread and rolls and I am able to sell it. The more customers I have or the more rolls and breads I am able to bake, to bigger is my effectiveness in the production of rolls and breads.

Efficiency comes in, when I try to do find out how much effort I put into the production. Do I need one work hour for baking one roll? That’s not very efficient and is not very profitable either. Do it cost me 2EUR for the baking of this one roll? It’s not very efficient, too. The question is with much effort is the effectiveness correlated?

Therefore, the effectiveness is just a measure of my business impact or the impact of everything else I do. But the efficiency is the ratio of the effectiveness to the effort I put into creating the effect.

In most situations the efficiency is more important than the effectiveness. The efficiency helps to be profitable. If a small business, which is not very effective (does not have a big market penetration), but is profitable enough to pay its bills and to invest enough, is healthy. But, the opposite is a company have a tremendous impact on the market (or maybe I am a market leader ;-)), but it does not make profit, then it’s just a question of time until it gets bankrupt.

Therefore: It’s better to have a small but profitable business than having a big business which struggles to make profit.

The Curse of Cp and Cpk

In factory automation the calculations of Cp (Process capability index) and Cpk (measure of process capability) values are a good tool to monitor and proof performance and quality. I do not give here the full theory for the calculation of Cp and Cpk and all the math behind it. I want to show some issues from my professional experience which arise in understanding and also in expectations.

Some Theory for Recapitulation

For a given process measurements are performed and for a single parameter the average (Avg) and the Standard Deviation (StdDev) are calculated. The Cp and Cpk values are calcualted in the following way:

Cp = (USL – LSL) / (6 * StdDev)

Cp,upper = (USL – Avg) / (3 * StdDev)

Cp,lower = (Avg – LSL) / (3 * StdDev)

Cpk = Min(Cp,upper, Cp,lower)

The Cp value gives the theoretical capability of the process. It assumes a centered production and evaluates the best possible outcome for the given process deviation. This is the target to reach for process centering. The Cp value it self should be maximized with minimizing process deviation.

The Cpk value gives the current process performance and takes also into account if the process is not centered. If the process is centered the equation Cp = Cpk is valid.

For a Six Sigma production, values of Cp=2.0 and Cpk=1.5 are expected. The theoretical process capability should show a process deviation which matches six times into the process specification. A normal variation of the process around its center is always present and can not be avoid completely. Therfore the long time Cpk value is allowed to be smaller due to experiences of process variations in field of 1.5 sigma around the target.

If everything is Six Sigma ready, one gets only 3.4 violations of the specification into each direction out of one million samples. That means 6.8 violations in total for one million samples.

The Issue of Normal Distribution

The first issue and seldom questioned is: „Are my parameters normally distributed?“

In reality I have seen processes with Cp and Cpk values smaller than expected or wanted, but without any fail. The reason is that the parameters used are not normally distributed and the caluculations and statistics go fail. One can think of a rectangular distribution which is always within specification, e.g. between LSL/2 and USL/2. The calculation of Cp and Cpk give a Sigma level which tells to have a messy process. But in reality everything goes fine and no violation is to be expected. Only the math gives numbers out of wrong expectations.

A gate oxid growth for example can not be normally distributed due to the impossibility of negative gate oxid thicknesses. Processes which are under SPC (Statistical Process Control) control where corrective actions are performed, can also not be normally distributed. The influence of the corrective actions destroy the normal distribution if it were present before-hand.

One should always do a test for normal distribution first before calculation Cp and Cpk. In reality only a few processes are quite well normally distributed and for them the normal Cp and Cpk measurements are suitable. In most processes one should use different distributions for calculations or better do a counting of violations and a calculation back to a Sigma level. Those gives better information about real processes.

Hunting for Cp and Cpk

In a lot of industries, quality is one of the most important values and in some even life saving like in aerospace, automotive and medical industry. The pressure to reach and proof quality is very strong and necessary to stay in business and to challenge ones competitors. Cp and Cpk values are crucial for all purposes.

Cp and Cpk strongly depend on the specifications used. This leads to the next section…

Setting the Limits

When it comes to customer satisfaction, good Cp and Cpk values are important goals. It’s sometimes hard to reach them technically, but choosing the right limits can help. I heard, that the specification limits can be adjusted to meet Cp and Cpk value requirments and that these limits do not have any restrictions due to the fact that the process is controlled trougth its control limits. I knew afterwards I have to write about the right limits. Even a professional consultant on this mentioned something like that and I could only explain why there are technical limits for process and quality engineering.

At first we should define which limits are available for process control and what the purposes of them are.

Functional Limits

These limits define within which limits the product is suspected to work. Outsite these limits the product is just waste, not to be used and therefore to be dumped. There is nothing more to say… 😉

Specification Limits

These limits are negotiated with the customers or internal limits which are defined to decide whether a product is to be delivered or not. Specification limits can be defined multiple times for different customers, markets or functions, because not all product purposes need the same quality. The only requirement is, that the specification limits are equal to the functional limits or smaller within the functional limit range. A product which does not work at all is not suitable for any market except for selling to waste recycling.

Control Limits

Control limits are used for triggering corrective actions. If the process runs out of the control limits, a production system should signal this event immeditially and someone responsible has to start corrective actions to get the process back to target or the deviation back to a normal level. The control limits need to be defined left and right (or above and below) the target to make sense for corrective actions. For the purpose to start corrective actions, the control limits also have to be defined between the specification limits. If the control limit triggers an event outsite specification, it’s too late for any meaningful actions. Several control limits can be defined if necessary to separated different levels of escalation or different levels of inversive corrective actions.

Relationship of Limits

Regarding the facts above there is a simple relationship between all these limits which can be written as:

UFL >= USL > UCL > TARGET > LCL > LSL >= LFL

Shortcut Name Meaning
UFL Upper Functional Limit The upper limit for the product to work properly
USL Upper Specification Limit The upper limit of the product specification as delivery criterium
UCL Upper Control Limit The upper control limit as trigger for corrective action
TARGET Target The production target and optimum goal for production
LCL Lower Control Limit The lower control limit as trigger for corrective action
LSL Lower Specification Limit The lower limit of the product specification as delivery criterium
LFL Lower Functional Limit The lower limit for the product to work properly

The Curse Revealed

Where is the curse now in all this? Everything is well defined and we do the right statistics!?

In reality, most parameters are not normally distributed and the calculation of Cp and Cpk give numbers implying a worse process than really present. The numbers do not look very well because of wrong asumptions of normal distributions. The correct calculation is difficult and only statistics based on event couting is really accurate, but the number of events for counting need to exceet the 1 million mark, which is not really practical.

Customers also want Cp and Cpk values which do have a well defined sigma level like 3 (Cp = 1.0), 4 (Cp = 1.33), 5 (Cp = 1.67) or 6 (Cp = 2.0). The specification limits can only be expanded to functional limits and the process has to have a deviation and variation to meet this goal which is sometimes not really meetable due to physical constraints of machines or processes. Sometimes the process capability can not be proofed accurately with Cp and Cpk values.

In semiconductor industry for example one has to deal with physical issues. For an 8nm tunnel oxid for example one has to grow a layer of roughly 30 atoms to meet the thickness. The specification is set to 8nm +/- 1nm. 1nm is roughly 4 atoms. To meet the Six Sigma level one has to have a process deviation of a 6th of this 4 atoms, which is 2/3 of an atom. Therefore, to meet the Six Sigma level one has to prepare with a process variation of 2/3 of an atom and that in a process which processes a whole 8 inch wafer. For the whole area only a 2/3 of an atom of variation is allowed. These kind of tunnel oxids are prepared in 0.6um technologies. The current technologies in CMOS go down to smaller than 0.03um and the tunnel oxids become thinner and thinner… We obviously meet the physical limitations here.

What can be done, when customers have their expectations and the physics is at its end? Sometimes there is a real gap between needs, expectations and physical capabilities.

Solution

Sometimes we have to slow down, to sit together and to discuss in detail and technically correct what’s going on. If the Cp and Cpk values are used for production control, it’s a good step, but the blind hunt for numbers is not leading to the final target: Quality. The numbers are not always correct and the expectations have to be adjusted to the reality. Every production process should be optimized for best possible and effortable quality, but this has to be made transparent to external and internal customers of manufactoring parameters.

For managers, engineers and customers, there should be open discussions and if needed trainings to get background knowledge to get into constructive discussions and decision making procedures. Otherwise there is a large potential for professional conflicts…

Layer Model of Software Development Responsibility

During my professional life, I often have to explain my point of view of software development responsibilities. When, who has and where to develop software? Does a factory or manufacturer have to develop its own software? Is there not a standard software already around which is more expensive, but already present and a responsible software vendor is out there in case of bugs? Do I need a own software development branch? Could an external system house or software developer develop my specific solutions?

To make this tough question even more difficult, let me give you the straight answer: „It depends…!“ 🙂

Let me explain my own small model about the responsibility layers within a virtual high tech factory. These layers can be identified within every organization with some differences, but the principles stay the same.

In reference to the OSI model, I created a small model for explaining these issues. There are roughly five or six layers within the IT landscape within a factory.

The Layers of Responsibility

Layers of Responsibility
Layers of Responsibility

Layer 1: Hardware Layer

This layer is the typical machine which is bought from a vendor. The machine is put and installed in the factory and does its job. The responsibility is clear: The machine vendor is responsible for its machine to work. The own facility crew has to assure that all media are available, but the machine is under vendor’s responsibility.

Layer 2: Data Collection/ Measurement Layer

Within this layer, data is collected or measured for later usage in layers above. A lot of machines have standard interfaces for presenting information outside, but some information is still not visible per default and can only be parsed out of log files or got from other sources.

As long as the data comes from standard interfaces from the machine, the machine vendor is fully responsible to get the interface work as documented. As soon as there is no standard way the software for getting these information must be customized.

Layer 3: Data Preconditioning and Conversion

In this layer, data coming from layer 2 is preconditioned and converted into formats and data which is more suitable for layers above. Every company, factory or organization needs other data and with another grade of detail. To reduce the amount of data, uninteresting data has to be eliminated and the grade of detail can be reduced by statistical summarizing. This layer is very tightly coupled with layer 2.

Some machines have the possibility to precondition data, but in most cases the amount and the grade of detail for data are to be customized. This layer is under full responsibility of the customer and the customer has to think about ways to do this customization.

Layer 4: Data Storage

Collected data needs to be stored for later use in a flexible and reliable way. This storage should be done in standard tools like databases and file servers. The responsibility is at the vendors of this storage systems. It would be strange if a non-software company would develop its own database systems and file servers.

Layer 5: Calculation

The data stored should now be used meaningfully. To achieve these parameters and numbers have to be calculated to give the appropriate information. Due to a customized table space or server landscape where the data is stored and a customized definition of parameters and measurement values those calculations have to be customized, too. The locations of data and the detailed definitions of parameters vary from organizations, factories and companies to others. This layer is to be customized in responsibility of the own company.

Layer 6: Presentation

The presentation could and should be done in standard software. It does not matter in first place whether it is within an office suite or with dedicated reporting tools. The views within these tools have to be customized and this should be part of layer 5. The view itself is done with standard software and responsible is therefore the vendor of the used software.

Customization

As shown above, a lot of software is around to solve standard tasks like storage and presentation. The machine vendors should ship interfaces to get all interesting data out of the machines. Everything else could and should be customized due to varying forms of server landscapes, database schemes and grades of details. Every company, factory or organization has its own ideas on how to collect data and which grade of detail and what amount.

Therefore, in layers 2, 3 and 5, customization is needed and should be done in responsibility to the company using the software. There is no other way around. Machine vendors do not know the details of their customers and can not meet all demands. They can only create easy to use, reliable and well documented interfaces to get all important information out of the machine. Everything else has to be customized.

The question „Who should do the development?“ is more difficult to answer. My answer would be: „If you can effort it, do it on your own. Hire at least two people who can program the software for the layers and be happy about the knwoledge and capability to make changes on your own.“

If your company, factory or organization is large enough to keep two or more programmers busy the whole time, you should hire them on your own and make the software on your own:

    • External software developers are more expensive due to the extra money they need for offices and administration.
    • In case of bugs, your own programmers know your company better and can fix the bugs easily.
    • In situations of change, your own people can make the needed changes. That’s faster and cheaper. I do not only speak about changing a company logo after a merger, but about changing a whole product lines or factories to new machines or products.

Summary

There are some obvious parts within an IT landscape where customization is needed and should be done: Data collection, measurement, pre-condition, conversation and final calculation. If it’s affordable, one should do it on its own to get the best benefit out of it and to be flexible in situations of change.

The Curse of GOTO

When I started to learn programming professionally in 1991 at the Schülerrechenzentrum Dresden I was always told: „Do not use GOTO statements! It is never needed due to other possibilities which are available. GOTOs do disturb the program flow, the understanding of the program becomes more difficult and the program is not well structured anymore.“ I started to program some years before that on a C64 and there was practically no other way than using GOTOs and therefore, it sounded strange to me at first.

At the Schülerrechenzentrum Dresden I learned to program Turbo-Pascal and I was taught to use never ever GOTOs, but to think about a good program flow and to implement decisions and jumps with IF statements and calls of functions and procedures. Over the years I always followed this advice, but from time to time I started thinking about the usage again, but I never found a situation where a GOTO statement would be a better choice than other possibilities of implementation.

Following some facts and taught why not to use GOTOs.

Complexity

GOTOs increase source code’s complexity or seem to increase it. If there is a label in source one never knows exactly what GOTOs and how much will jump there in which circumstances and where are they located. If there is a GOTO around, the search for the corresponding label starts. The flow is not obvious any more and a label is not always distinct.

In the name of complexity GOTOs should be replaced by IF statements and function calls. These are easy to understand and with the right indentation and naming it’s quite obvious what the program does and what the intentions are.

Maintainable Code and Human Understanding

Try to understand code which is written with GOTOs. You always have to find out, where the fitting label is and under which circumstances the GOTO is invoked and what the current status of all variables are. It becomes more difficult in languages where a GOTO is allowed to jump out of the current function into another one or where the GOTO is allowed to jump into loops from outside. It’s very difficult to see and understand what the initial values are after the jump and how the program will proceed. A GOSUB and a RETURN is the same mess.

„Any fool can write code that a computer can understand. Good programmers write code that humans can understand. „
– Martin Fowler –

For maintainable code, source code is needed without such difficulties. A clean source code exposes all its conditions and its program flow. A GOTO messes this up and should therefore never used.

Refactoring

In Martin Fowler book „Refactoring – Improving the Design of Existing Code“ a lot of techniques and patterns are described for improving and developing good code. Refactoring, improving the code, can only be done, if the program flow is obvious and the behavior can be foreseen for any change. If a GOTO is within the area of the refactored code, the change or move of the corresponding label can dramatically change the programs behavior and it’s not that obvious in the first place.

In the name of maintainable code, refactoring should be possible. GOTOs lead to source code which is not easily refactorable and it should therefore be avoided under any circumstance.

For further information about clean code, have a look to the book Clean Code by Robert C. Martin.

Pythagorean Triangle of Life

Pythagorean Triangle of Life

„Concern should drive us into action and not into a depression. No man is free who cannot control himself. „
– Pythagoras –

The Pythagorean Triangle of life is a great symbol and concept how to reach harmony in a society. A society is a complex system of people trying to life together in harmony. The Pythagorean Triangle describes how it should be achieved.

Pythagoras was fascinated by triangles. He found the triangles are a universal concepts to describe nature. The number three was the number of harmony in his tetraktis. The tetraktis is a picture of ten dots arrange in a triangular shape. This image represented the essence of Pythagoras’s teachings.

The whole game or drama of life is playing between three points, Pythagoras mentioned. The first point is the self. With self he understood our spiritual self, the soul. The body is not part of that and already belongs to the second point: the life. Life is the material existence outside of the soul.

In our society we mainly only with these two points. The number two for Pythagoras was the number for duality, opinion, choice and the potential between two different counterparts. By arguing about opinions disharmony is created. For Pythagoras the number two was the number of disharmony, too. When we deal only with self and life, we create large disharmony in our society as can be currently observed in our daily life.

It is to be mentioned that psychology works only with these two points and the interaction of human beings. The so-called Take-And-Control-Model is dealing with these two points, too. The self focuses on life, because it wants to control something. When someone tries to control another one else then it is an intervention into the self-sovereignty and self-determination of an individual. This influences the freedom of an individual and it has therefore the natural right to defend that. The intervention arises out of a strong need. To fulfill a need one wants to take it, but if it is taken without permission, then someone feels a loss. Does it sound familiar?

Common sayings in our time are „What you want to have, you have to take.“ and „When you want something happen, make it happen.“. That’s exactly the Take-And-Control-Model. That leads to disharmony.

What we often forget is the third point of this model, which got completely out of sight in our so-called modern time. The third spot is theos in Greek and means god. The ancient word for god also means divine truth. Life< and divine truth are connected directly. „The truth will reveal itself through life“ is common saying. We are connected with the divine truth. We experience this connection as intuition, as feelings in our heart or belly. Sometimes we know, that somebody wants to phone us, we have experiences of deja-vus and so forth. The way of harmony is to serve the divine truth, to act and give in its name and to be rewarded for that through life. „Give and you will be given.“ This sounds quite familiar for many people with different backgrounds and traditions. Expressed in a more general way: When we serve the truth, we will receive whatever we need through life.

Some people don’t like the words serve and give, because it sounds like work and surrender. Giving in name of divine truth can also be understood as acting truthfully, being natural, honest and without any kind of affectation.

„Virtue is harmony. „
– Pythagoras –

Hook Scripts Revised

Hook scripts are flexible in their implementation. I want to show an easy way how to overcome the obstacle of system routines which contain high volatile code due to environmental changes like company changes, mergers and other changes within the logic of any system.

Definition

Hook scripts are normally scripts which are started in special situations on a server or within a software system. Hook scripts can be used to do some customized work during a process in an standard environment, they can be used to trigger other external systems or do a lot of other work which could no be done within a closed systems. Therefore, hook scripts are a great way to provide customers well defined points in a system which can be customized for special needs. I strongly recommend to consider hook scripts as customization solution for larger software systems. A good example are Subversion’s hook scripts.

Issue

During my professional life I faced several situations were I already knew that one or more parts of my new designed and implemented system were to be changed some weeks later on and that this might not be the last time a change was necessary. One time, I faced this situation shortly after a merger of my company. We had to change the convention of lot numbers to the new company standard. We had to face a transition time when we were forced to handle two different types of lot number which were totally different in nomenclature. What could be done here? I designed a system in C/C++ with approximately 100k lines at this time. I didn’t want to implement the reason for the next minor release straight into the system.

Solution

My idea to solve this situation was to use hook scripts to do jobs which perform tasks for volatile specifications and for volatile production environments which are characterized by continuous improvements and permanent updates and optimizations. I had this idea some years before for a part of the system which was also threatened by a volatile production environment. Volatile environments mean a permanent potential reason of change in any software system. I am a lazy guy and therefore, I did not want to make the changes deep within the C/C++ code every time again.

To stay at the example from above, I had designed my system in a way that lot number conventions did usually not matter. A lot number was equal to another if the string comparison was equal. Plain and easy. But I faced the need for a conversion of split lot numbers to root lot numbers. The conversion for split lot numbers also did change. The old convention we had were plain numbers for lots like „nnnnnnn“ (n = decimal number). A zero in the 7th position was for root lots and was changed for splits to other decimal number. The new convention was „lnnnnn“ (l = letter, n = decimal number) and split lots were marked with additional letters at the end, changing even the length of the lot number. I had to face the situation that I needed a method which could translate any lot number to root lot numbers…

The implementation was done in C/C++ with a simple call to POSIX’s system command which called a script which was stored within my normal shared system files somewhere at /usr/share/. The script was called hook_split2rootlot.pl. I implemented a simple logic with less than 20 lines there for the simple translation. Everytime I have to change the convention of translating these lot numbers I just need to change this little and easy to understand script and ready is my work. Due to the implementation in Perl, it is very easy to add a more sophisticated logic for other purposes, too.

Trade-off

A serious trade-off is only speed due to POSIX system calls are quite expensive in time. Hook scripts should only be used for functions which are not often called. For system parts which run very often and have to be fast another solution is to be implemented.

Summary

Hook scripts, if implemented in the right way, can make a system very flexible, customizable and also easy to administrated. I recommend to consider hook scripts during architectural development as a possible solution for flexibility.