Finding and slaughtering inactive bugs in inserted programming is a troublesome business. Chivalrous endeavors and costly instruments are frequently required to follow in reverse from a watched crash, hang, or other unplanned run-time conduct to the underlying driver. In the most dire outcome imaginable, the main driver harms the code or information in such an unobtrusive manner, that the framework despite everything seems to work fine or generally fine-for at some point before the breakdown.
Over and over again builds quit any pretense of attempting to find the reason for rare abnormalities – on the grounds that they can’t be handily repeated in the lab- – excusing them as “client blunders” or “glitches.” Yet these phantoms in the machine live on.
So here’s a manual for the most continuous underlying drivers of hard to-duplicate firmware bugs.
#10: Jitter
Some continuous frameworks request not just that a lot of cutoff times be constantly met yet additionally that extra planning requirements be seen simultaneously. For example, overseeing jitter.
A case of jitter is appeared in Figure 1. Here a variable measure of work (blue boxes) must be finished before each 10 ms cutoff time. As represented in the figure, the cutoff times are completely met. Be that as it may, there is significant planning variety starting with one run of this activity then onto the next. This jitter is unsatisfactory in certain frameworks, which ought to either begin or end their 10 ms runs all the more accurately.
 |
Figure 1. A case of jitter in the planning of a 10ms errand
In the event that the work to be performed includes examining a physical information signal, for example, perusing a simple to-advanced converter, it will frequently be the situation that an exact inspecting period will prompt higher exactness in inferred esteems. For instance, varieties in the between test season of an optical encoder’s heartbeat check will bring down the exactness of the speed of a connected revolution shaft.
Best Practice: The most significant single factor in the measure of jitter is the overall need of the errand or ISR that executes the intermittent conduct. The higher the need the lower the jitter. The intermittent peruses of those encoder beat includes should subsequently ordinarily be in a clock tick ISR instead of in a RTOS task.
Figure 2 shows how the time period distinctive 10 ms of repeating tests may be affected by their relative needs. At the most noteworthy need is a clock tick ISR, which executes decisively on the 10 ms span. (Except if there is higher need interferes, obviously.) Below that is a high-need task (TH), which may in any case have the option to meet a common 10-ms start time exactly. At the base, however, is a low need task (TL) that has its planning enormously influenced by what goes on at higher need levels. As appeared, the stretch for the low need task is 10 ms +/ – roughly 5 ms.
 |
Figure 2. The effect of priority on jitter
#9: Incorrect Priority Assignment
Get your needs straight! Or on the other hand endure the result of missed cutoff times. Obviously, I’m talking here about the general needs of your ongoing assignments and interfere with administration schedules. In my movements around the installed structure network, I’ve discovered that most ongoing frameworks are planned with specially appointed needs.
Lamentably, mis-organized frameworks regularly “show up” to work fine without detectably missing basic cutoff times in testing. The most pessimistic scenario remaining task at hand may have never yet occurred in the field or there is adequate CPU to coincidentally prevail in spite of the absence of appropriate arranging. This has lead to an age of implanted programming engineers being unconscious of the correct procedure. There is just too little criticism from non-reproducible cutoff time misses in the field to the first structure group—except if a passing and a claim power an investigation.
Best Practice: There is a science to the way toward doling out relative needs. That science is related with the “rate monotonic calculation,” which gives a standard method to relegate task needs dependent on realities. It is likewise connected with the “rate monotonic examination,” which causes you demonstrate that your effectively organized errands and ISRs will discover adequate accessible CPU transfer speed between them during incredibly bustling outstanding burdens called “transient over-burden.” It’s really awful most designers don’t have a clue how to utilize these instruments.
There’s lacking space in this segment for me to clarify why and how RMA functions. In any case, I’ve composed on these points previously and prescribe you start with Introduction to Rate-Monotonic Scheduling and afterward read 3 Things Every Programmer Should Know About RMA.
It would be ideal if you realize that on the off chance that you don’t utilize RMA to organize your undertakings and ISRs (as a set), there’s just a single substance with any ensures: the one most noteworthy need errand or ISR can take the CPU for itself at any active time—notwithstanding need reversals!— and hence has up to 100% of the CPU data transfer capacity accessible to it. Additionally, note that there is no dependable guideline about what level of the CPU data transfer capacity you may securely use between a lot of at least two runnable except if you do follow the RMA plot.
#8: Priority Inversion
A wide scope of awful things can turn out badly when at least two undertakings arrange their work through or in any case share, a singleton asset, for example, a worldwide information region, store article, or fringe’s register set. In the initial segment of this section, I portrayed two of the most widely recognized issues in task-sharing situations: race conditions and non-reentrant capacities. Be that as it may, asset imparting consolidated to the need based appropriation found in business continuous working frameworks can likewise cause need reversal, which is similarly hard to repeat and troubleshoot.
The issue of need reversal comes from the utilization of a working framework with fixed relative undertaking needs. In such a framework, the developer must relegate each undertaking it’s need. The scheduler inside the RTOS gives an assurance that the most elevated need task that is prepared to run gets the CPU—consistently. To meet this objective, the scheduler may acquire a lower-need task in mid-execution. In any case, when errands share assets, occasions outside the scheduler’s control can in some cases forestall the most elevated need prepared undertaking from running when it should. At the point when this occurs, a basic cutoff time could be missed, making the framework come up short.
In any event three assignments are required for a need reversal to really happen: the pair of most elevated and least relative need must share an asset, state by a mutex, and the third should have a need between the other two. The situation is constantly appeared in the figure beneath. To start with, the low-need task obtains the mutual asset (time t1). After the high need task seizes low, it next attempts yet neglects to secure their common asset (time t2); control of the CPU returns back to low as high squares. At last, the medium need task—which has no enthusiasm at all in the asset shared by low and high—acquires low (time t3). Now the needs are transformed: the medium is permitted to utilize the CPU for whatever length of time that it needs, while high sits tight for low. There could even be various medium need undertakings.
 |
Figure 3. An example of priority inversion
The hazard with need reversal is that it can forestall the high-need task in the set from fulfilling an ongoing time constraint. The need to comply with time constraints regularly goes connected at the hip with the decision of a preemptive RTOS. Contingent upon the final result, this missed cutoff time result may even be dangerous for its client!
One of the significant difficulties with need reversal is that it’s commonly not a reproducible issue. To begin with, the three stages need to occur—and in a specific order. And afterward the high need task needs to really miss a cutoff time. Either of these might be uncommon or difficult to imitate occasions. Lamentably, no measure of testing can guarantee they won’t ever occur in the field.
 |
Figure 4. A simple design pattern with clear memory buffer ownership
Best Practice: There is a basic method to maintain a strategic distance from memory spills and that is to plainly characterize the proprietorship pattern or lifetime of each kind of stack designated object. The figure above gives one regular possession pattern including supports that are designated by a maker task (P), sent through a message line, and later decimated by a buyer task (C). To the most extreme degree conceivable this and other safe plan patterns ought to be followed progressively frameworks that utilization the pile.
Notwithstanding keeping away from memory releases, the plan pattern appeared in Figure 4 can be utilized to guarantee against “out-of-memory” mistakes, in which there are no supports accessible in the cushion pool when the maker task endeavors an allotment. The strategy is to (1) make a committed cradle pool for that kind of portion, say a cushion pool of 17-byte supports; (2) use lining hypothesis to suitably measure the message line, which guarantees against a full line; and (3) size the cushion pool so there is at first one free cradle for every purchaser, every maker, in addition to each opening in the message line.
source:-https://barrgroup.com/Embedded-Systems/How-To/Top-Ten-Nasty-Firmware-Bugs