AVR-Chat

I am about to use the WDT for the first time (ATMega16).  I do not have a
software problem in my device at the moment but I want to have the WDT
active in my production application.

From reading the full datasheet for the ATMega16 it seems that, after a WD
reset, there is no way to determine what the value of the Program Counter
was at the time.

So how does one record information to determine where the bug might be in a
situation where it is very difficult to reproduce the conditions that caused
the software hang?

I can have the application leave 'breadcrumbs' at different points but this
would either take a lot of time to narrow in on or take a lot of
'breadcrumb' code.

Chuck -

My experience is that a watchdog timer is simply a reset of last  
resort, after the horses are out of the barn, but hopefully before  
they have wandered to some place dangerous. I know of no way to read  
the watchdog timer. You only know when it has timed out.

I personally don't try to keep a bread crumb trail. I'm just thankful  
that it got reset. I have not had a known watchdog reset in recent  
history or ever with an AVR so I don't know what I would do. Of  
course, that does not mean that my next project will be so fortunate!  
Or, the one after that.

Jim Wagner
Oregon Research Electronics.

On Jul 20, 2009, at 7:52 PM, Chuck Hackett wrote:

> I am about to use the WDT for the first time (ATMega16). I do not  
> have a
> software problem in my device at the moment but I want to have the WDT
> active in my production application.
>
> From reading the full datasheet for the ATMega16 it seems that,  
> after a WD
> reset, there is no way to determine what the value of the Program  
> Counter
> was at the time.
>
> So how does one record information to determine where the bug might  
> be in a
> situation where it is very difficult to reproduce the conditions  
> that caused
> the software hang?
>
> I can have the application leave 'breadcrumbs' at different points  
> but this
> would either take a lot of time to narrow in on or take a lot of
> 'breadcrumb' code.
>
>
> 



[Non-text portions of this message have been removed]

Just a minor point, the newer range of AVR devices , (atmega168, xmega etc) have the capability of mapping the watchdog timeout to an interrupt rather than a reset, therfore breadcrumbs can be managed.

2 methods spring to mind with watchdogs,

1) The dark art of assertion Macros, these can prove inavluable in determining a porblem area of code.

2) Just before you "Tickle/Kick" the watchdog include a routine for writing a value to an eeprom area a bytye should do it, if his is a different value for each area of code then the value could be read on reset and a deduction made about which areaa of code caused the problem, his would add little latency, although would degrade the eeprom life, it could be implemented in debug only. 

How do you intent to deal with the watchdog reset after the code has been implemented as a product ? Do you record, inform, analyse or just let it resert and hope no one notices.

Are you using any real time time ? Context switching and re entrancy can prove a real nightmare to debug.

Common steps for RTOS systems are 

1) Build a watchdog task which is the only part of the software that is allowed to tickle the watchdog.

2) Create a data structure that has one entry / task. Whenever a task starts it increments this entry.

3) Increment the data "atomically".

4) Assess how many times/ sec a task "should " run, create min/max values for the data structure for each tasks, that way if a "spurious" number is detected allow the system to time out. 

This way you will get "a feel" for how the system really works, bottle necks etc and you adjust accordingly.

And remeber the best watchdog of all is theone that is external to the system and does not rely on software, external to the CPU and shares no resources.

Think carefully on how / what you are using/abusing the watchdog, it may give you more problems than it solves.

Regards


--- In AVR-Chat@yahoogroups.com, Jim Wagner <wagnerj@...> wrote:

>
> Chuck -
> 
> My experience is that a watchdog timer is simply a reset of last  
> resort, after the horses are out of the barn, but hopefully before  
> they have wandered to some place dangerous. I know of no way to read  
> the watchdog timer. You only know when it has timed out.
> 
> I personally don't try to keep a bread crumb trail. I'm just thankful  
> that it got reset. I have not had a known watchdog reset in recent  
> history or ever with an AVR so I don't know what I would do. Of  
> course, that does not mean that my next project will be so fortunate!  
> Or, the one after that.
> 
> Jim Wagner
> Oregon Research Electronics.
> 
> On Jul 20, 2009, at 7:52 PM, Chuck Hackett wrote:
> 
> > I am about to use the WDT for the first time (ATMega16). I do not  
> > have a
> > software problem in my device at the moment but I want to have the WDT
> > active in my production application.
> >
> > From reading the full datasheet for the ATMega16 it seems that,  
> > after a WD
> > reset, there is no way to determine what the value of the Program  
> > Counter
> > was at the time.
> >
> > So how does one record information to determine where the bug might  
> > be in a
> > situation where it is very difficult to reproduce the conditions  
> > that caused
> > the software hang?
> >
> > I can have the application leave 'breadcrumbs' at different points  
> > but this
> > would either take a lot of time to narrow in on or take a lot of
> > 'breadcrumb' code.
> >
> >
> > 
> 
> 
> 
> [Non-text portions of this message have been removed]
>

On Tue, Jul 21, 2009 at 4:01 AM,
s.holder123@btinternet.com<s.holder123@btinternet.com> wrote:
>
>
> Just a minor point, the newer range of AVR devices , (atmega168, xmega etc)
> have the capability of mapping the watchdog timeout to an interrupt rather
> than a reset,

Check the errata, that doesn't work in some parts.

A bad recovery can actually be more dangerous  that crashed code
if done wrong.

> 1) The dark art of assertion Macros, these can prove inavluable in
> determining a porblem area of code.

http://directory.fsf.org/project/nana/
may be better.

>
> 2) Just before you "Tickle/Kick" the watchdog include a routine for writing
> a value to an eeprom area a bytye should do it,

That could wear out a EEPROM block fast.  Even if just used in debugging mode.
You should always debug the code you ship.  Having debug mode and a ship mode
is not wise.


> 1) Build a watchdog task which is the only part of the software that is
> allowed to tickle the watchdog.

http://www.ganssle.com/watchdogs.pdf

-- 
http://www.softwaresafety.net/
http://www.wearablesmartsensors.com/
http://www.designer-iii.com/
http://www.unusualresearch.com/

> So how does one record information to determine where the bug might be in a
> situation where it is very difficult to reproduce the conditions that caused
> the software hang?
>
> I can have the application leave 'breadcrumbs' at different points but this
> would either take a lot of time to narrow in on or take a lot of
> 'breadcrumb' code.


I do this for some applications.
I have a task buffer and an ISR buffer, which are write-only circular buffers.
Any significant task that starts (not one that gets called and decides
not to run) puts an entry in the task buffer.
Any ISR that starts puts an entry in the ISR buffer.

On a reset, I can use the ice to breakpoint and read out the buffers.
In some apps, I've used the serial port to send "last task", "last
ISR" and "reason for reset" when it's waiting for the boot loader
magic word.

It all depends on what you need, and how much ram you can spend on it.


-- 
There is no computer problem which cannot be solved by proper
application of a sufficiently large hammer.

In a task:

#ifdef Task_Logging
	/// Indicate to "black box" what happened.
	Log_Task(105); //< BlackBox.c
#endif

The logger:

// Task log with optional timestamps using T2
// T2 is initted along with T0
// in peripherals.c
//
// GENERALLY, tasks don't log unless they execute, but this is
determined by where
// the logging call is inserted. Normally the logging call should be
inserted after
// all the conditions to run the task have been met, but before the
task actually runs.
// You may want to push the logging ahead, so you can see who gets
called, but that will
// fill the buffer very quickly.
//
#ifdef Task_Logging
void Log_Task(uint8_t Task_Code){

	Task_History[Task_History_Index++] = Task_Code;		
		if ((Task_History_Len - 1) < Task_History_Index) {
			Task_History_Index = 0;
		}

#ifdef Log_Timestamp
	// Optionally, log a timestamp too
	Task_History[Task_History_Index++] = TCNT2; //Int_Counter;		
		if ((Task_History_Len - 1) < Task_History_Index) {
			Task_History_Index = 0;
		}
#endif
}
#endif


The buffer it goesinta and its pointer.

#ifdef Task_Logging
	// Write-only circular buffer showing tasks
	uint8_t		Task_History[Task_History_Len];
	uint8_t		Task_History_Index;
#endif


-- 
There is no computer problem which cannot be solved by proper
application of a sufficiently large hammer.

Thanks for the tip on the nana lib, could come in useful, but i do have some tried and tested assertion macros that have served me fine,.

I wasn't suggesting that the eeprom fix is a good idea for production code, only to find out what is going wrong, you need some area that won't be over written when a reset/error occurs that you can recover.
unless you declare some memory as noinit, then run a reset task before the main function runs, read out the values then clear it to a known value.

Yes it can wear out fast (Rotating addresses sometimes helps)
Its a debug tool only !!

As for mapping the wdt to an interrupt, haven't come across any problems yet with it, this way you could use ram as the the scratchpad rather then eeprom, as at least the errors could be trapped outputted here.

Regards


--- In AVR-Chat@yahoogroups.com, Bob Paddock <bob.paddock@...> wrote:

>
> On Tue, Jul 21, 2009 at 4:01 AM,
> s.holder123@...<s.holder123@...> wrote:
> >
> >
> > Just a minor point, the newer range of AVR devices , (atmega168, xmega etc)
> > have the capability of mapping the watchdog timeout to an interrupt rather
> > than a reset,
> 
> Check the errata, that doesn't work in some parts.
> 
> A bad recovery can actually be more dangerous  that crashed code
> if done wrong.
> 
> > 1) The dark art of assertion Macros, these can prove inavluable in
> > determining a porblem area of code.
> 
> http://directory.fsf.org/project/nana/
> may be better.
> 
> >
> > 2) Just before you "Tickle/Kick" the watchdog include a routine for writing
> > a value to an eeprom area a bytye should do it,
> 
> That could wear out a EEPROM block fast.  Even if just used in debugging mode.
> You should always debug the code you ship.  Having debug mode and a ship mode
> is not wise.
> 
> 
> > 1) Build a watchdog task which is the only part of the software that is
> > allowed to tickle the watchdog.
> 
> http://www.ganssle.com/watchdogs.pdf
> 
> -- 
> http://www.softwaresafety.net/
> http://www.wearablesmartsensors.com/
> http://www.designer-iii.com/
> http://www.unusualresearch.com/
>

One quick way that I'm fond of is to write to an external one-byte buffer.
 I usually don't have the I/O lines to do a parallel write, but use a
serial-in, parallel out part.  Before every task spit out a byte that
identifies what task is currently running. If the unit crashes I can look
at what is in that buffer - If one has infinite resources you could put
LEDs on the output so you have a visual indication, if you want to make it
REALLY robust only allow a certain range of values to be written so a
random call of the function won't write garbage out (if your code is OTL.)

Sure, it takes up code and board space though.  If you have a RTC on your
board you could use the static RAM in the part for your storage of the
debug/breadcrumb byte.  But using I2C for this will take a LOT of time and
might not be a great idea.

Just a thought,
DLC

>> So how does one record information to determine where the bug might be
>> in a
>> situation where it is very difficult to reproduce the conditions that
>> caused
>> the software hang?
>>
>> I can have the application leave 'breadcrumbs' at different points but
>> this
>> would either take a lot of time to narrow in on or take a lot of
>> 'breadcrumb' code.
>
>
> I do this for some applications.
> I have a task buffer and an ISR buffer, which are write-only circular
> buffers.
> Any significant task that starts (not one that gets called and decides
> not to run) puts an entry in the task buffer.
> Any ISR that starts puts an entry in the ISR buffer.
>
> On a reset, I can use the ice to breakpoint and read out the buffers.
> In some apps, I've used the serial port to send "last task", "last
> ISR" and "reason for reset" when it's waiting for the boot loader
> magic word.
>
> It all depends on what you need, and how much ram you can spend on it.
>
>
> --
> There is no computer problem which cannot be solved by proper
> application of a sufficiently large hammer.
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


-- 
Dennis Clark
TTT Enterprises