AVR-Chat

The GCC compiler optimiser seems to be doing a weird thing to me... I'd like to understand it. it seems to require a volatile qualifier when I am not expecting it.

I have the following variables defined
unsigned char LedLevel0=0;
unsigned char LedLevel1=0;
volatile unsigned char AdcVal[4];

AdcVal holds readings from the ADC and is loaded in an interrupt routine

main loop:

	//the main loop
	while(1)
	{
		LedLevel0=AdcVal[0];
		LedLevel1=AdcVal[1];
	}

compiler output:

	//the main loop
	while(1)
	{
		LedLevel0=AdcVal[0];
 3ea:	80 91 86 01 	lds	r24, 0x0186
		LedLevel1=AdcVal[1];
 3ee:	80 91 87 01 	lds	r24, 0x0187
 3f2:	fb cf       	rjmp	.-10     	; 0x3ea <main+0x8>

i.e. it's loading the AdcVal into r24, but never stores it into the other variable.

if I change the main loop to add a call to a delay routine then it works correctly. Also if I declare LedLevel0/1 as volatile.
But in my mind it shouldn't need to be declared volatile, nothing else is changing it.

	//the main loop
	while(1)
	{
		LedLevel0=AdcVal[0];
		LedLevel1=AdcVal[1];
		TimeWait(1);
	}

compiler output:
	//the main loop
	while(1)
	{
		LedLevel0=AdcVal[0];
 3ea:	80 91 86 01 	lds	r24, 0x0186
 3ee:	80 93 06 01 	sts	0x0106, r24
		LedLevel1=AdcVal[1];
 3f2:	80 91 87 01 	lds	r24, 0x0187
 3f6:	80 93 07 01 	sts	0x0107, r24
		TimeWait(1);
 3fa:	81 e0       	ldi	r24, 0x01	; 1
 3fc:	90 e0       	ldi	r25, 0x00	; 0
 3fe:	f4 cf       	rjmp	.-24     	; 0x3e8 <main+0x6>


-- 
Tim Mitchell

Tim,
 
The AdcVals are being changed in a intterupt routine then ? The compiler needs to know that the values being stored or where they are being stored can changed or else it will optimise incorrectly, i.e it needs to have foresight of the fact that the values can change, if a value can be change from an interrupt routine and it is declared globally then it needs to be declared as volatile if it is the stored variable or the changed variable.
 
In the first instance of your code because there is no other code in the main routine the compiler will see this as one variable = another variable and sets it at compile time as it cannot see anything ever changing the variable, when you add the delay in or some other code it has to store them in the required variable because it may need the registers that it is using.
 
Generally if you do have variables that are global and can be changed outside the main routine then declare them as volatile (at the expense of variable optimisation) or cast it to volatile when required.
 
*(volatile) unsigned char *)&LedLevel0 = AdcVal[0];
 
Regards

 

________________________________

From: Tim Mitchell <tim@sabretechnology.co.uk>
To: AVR-Chat@yahoogroups.com 
Sent: Friday, 13 April 2012, 11:35
Subject: [AVR-Chat] gcc compiler bad behaviour


  

The GCC compiler optimiser seems to be doing a weird thing to me... I'd like to understand it. it seems to require a volatile qualifier when I am not expecting it.

I have the following variables defined
unsigned char LedLevel0=0;
unsigned char LedLevel1=0;
volatile unsigned char AdcVal[4];

AdcVal holds readings from the ADC and is loaded in an interrupt routine

main loop:

//the main loop
while(1)
{
LedLevel0=AdcVal[0];
LedLevel1=AdcVal[1];
}

compiler output:

//the main loop
while(1)
{
LedLevel0=AdcVal[0];
3ea: 80 91 86 01 lds r24, 0x0186
LedLevel1=AdcVal[1];
3ee: 80 91 87 01 lds r24, 0x0187
3f2: fb cf rjmp .-10 ; 0x3ea <main+0x8>

i.e. it's loading the AdcVal into r24, but never stores it into the other variable.

if I change the main loop to add a call to a delay routine then it works correctly. Also if I declare LedLevel0/1 as volatile.
But in my mind it shouldn't need to be declared volatile, nothing else is changing it.

//the main loop
while(1)
{
LedLevel0=AdcVal[0];
LedLevel1=AdcVal[1];
TimeWait(1);
}

compiler output:
//the main loop
while(1)
{
LedLevel0=AdcVal[0];
3ea: 80 91 86 01 lds r24, 0x0186
3ee: 80 93 06 01 sts 0x0106, r24
LedLevel1=AdcVal[1];
3f2: 80 91 87 01 lds r24, 0x0187
3f6: 80 93 07 01 sts 0x0107, r24
TimeWait(1);
3fa: 81 e0 ldi r24, 0x01 ; 1
3fc: 90 e0 ldi r25, 0x00 ; 0
3fe: f4 cf rjmp .-24 ; 0x3e8 <main+0x6>

-- 
Tim Mitchell 




[Non-text portions of this message have been removed]

--- In AVR-Chat@yahoogroups.com, STEVEN HOLDER <s.holder123@...> wrote:
> Generally if you do have variables that are global and can be
> changed outside the main routine then declare them as volatile
I don't think this statement describes the issue precisely enough.  Firstly, the phrase "can be changed outside the main routine" is too broad because it includes normally called routines.  Secondly, the mention of "the main routine" may lead one to believe that the issue only applies to variables accessed in main().

The volatile attribute must be used on any variable that may be modified by a means that cannot be anticipated by the compiler given the code that it can see.  Examples include:
- variables representing hardware registers
- variables in multi-port memory
- variables modified by an interrupt handler
- variables modified by another thread/task/process that may run at any time

You may safely omit the volatile attribute (or cast it away) in cases where you are guaranteed that the variable cannot change unexpectedly, e.g. the variable is modified by an interrupt handler but interrupts are disabled.

It should be further noted that use of the volatile attribute does *not* guarantee atomic access.  This is an important consideration when working with variables wider than the data path (for 8-bit AVRs, multi-byte variables) and also for read-modify-write operations.

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

> The volatile attribute must be used on any variable that may be modified
> by a means that cannot be anticipated by the compiler given the code that it
> can see. Examples include:
> - variables representing hardware registers
> - variables in multi-port memory
> - variables modified by an interrupt handler

I agree with those.

> - variables modified by another thread/task/process that may run at any
> time

Don't agree with that one.  Volatile is never the solution to a
problem with threads.
Either proper locks, or message passing, needs to be used.

> You may safely omit the volatile attribute (or cast it away)

I'd think really hard about casting it away as well.

> It should be further noted that use of the volatile attribute does *not*
> guarantee atomic access.

Hence my comment above.

>This is an important consideration when working
> with variables wider than the data path (for 8-bit AVRs, multi-byte
> variables) and also for read-modify-write operations.

Here is a real example for Tim:

Three different methods to read a 32-bit value changed inside
an interrupt on an 8-bit part.

/* Running on 8-Bit AVR: */

#include <avr/interrupt.h> /*
http://www.nongnu.org/avr-libc/user-manual/group__avr__interrupts.html
*/
#include <atomic.h>        /*
http://www.nongnu.org/avr-libc/user-manual/group__util__atomic.html
*/

static uint32_t volatile system_ticks_u32;
static uint32_t miss_match_count_u32;

ISR( some_timer )
{
 ++system_ticks_u32;
}

/*
 * In this example the 8-bit AVR changes the 32-bit system_tick variable
 * inside the interrupt, a bad value will be returned if
 * system_tick_get_BAD() called simultaneously.
 */

uint32_t system_ticks_get_BAD( void );
uint32_t system_ticks_get_BAD( void )
{
 return( system_ticks_u32 );
}

/* Interrupts are disabled for the shortest possible time: */
uint32_t system_ticks_get_GOOD( void );
uint32_t system_ticks_get_GOOD( void )
{

 uint32_t current_ticks_u32;

 /*
  * Interrupts turned off while copying 32-bit
  * value 8-bits at a time:
  */
 ATOMIC_BLOCK(ATOMIC_RESTORESTATE)
 {
  current_ticks_u32 = system_ticks_u32;
 }

 return( current_ticks_u32 );
}

/* Never turns the interrupts off: */
uint32_t system_ticks_get_BETTER( void );
uint32_t system_ticks_get_BETTER( void )
{

 uint32_t t1_u32, t2_u32;

 do{
  t1_u32 = system_ticks_u32;
  t2_u32 = system_ticks_u32;

  if( t1_u32 != t2_u32 )
  {
   ++miss_match_count_u32;
  }
 }while( t1_u32 != t2_u32 );

 return( t2_u32 );
}

Running on a real project of mine, using a XMega128A1 at 4.0 MHz,
miss_match_count_u32 is incremented about every twenty minutes, generally
in clusters of three each.  Not shown here is an other counter that
counts if the do{}while() ever repeats the loop more than once, which
has yet to happen.


--
http://blog.softwaresafety.net/
http://www.designer-iii.com/
http://www.wearablesmartsensors.com/

--- In AVR-Chat@yahoogroups.com, Bob Paddock <bob.paddock@...> wrote:
> Volatile is never the solution to a problem with threads.
Nonsense.  The volatile attribute is precisely the solution for *any* situation where a variable can be changed by an outside actor - even if it is another task.  As I noted, volatile doesn't address atomicity of access nor (as you alluded to) does it address serialization of resource access.  Those are different problems with different solutions.

Of course, you may argue that you should not design your system so that one task sits in a loop waiting for the value of a single-byte variable to be changed by another task.  That's a different discussion altogether.  If you do use this strategy, however, that variable *must* be volatile.


--- In AVR-Chat@yahoogroups.com, Bob Paddock <bob.paddock@...> wrote:
> I'd think really hard about casting [volatility] away as well.
Indubitably.  Tools of all kinds have proper and improper uses and may be dangerous if improperly used.  It is not dangerous, for example, to cast away volatility in an ISR if no any other actor can change the variable in question.  Whether the benefit of doing so outweighs the "costs" is a tradeoff decision like many others that must be made. 

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

--- In AVR-Chat@yahoogroups.com, Bob Paddock <bob.paddock@...> wrote:
>> The volatile attribute is precisely the solution for *any* 
>> situation where a variable can be changed by an outside actor
> No. http://blog.regehr.org/archives/28 "Nine ways to break your
> systems code using volatile"  See #8.
We may be arguing different points.  Consider the code below with two tasks in a preemptive task switching environment.  I don't claim that one should write task code like this.  I do claim that if one does write code like this in such an environment, myFlag *must* be volatile.

uint8_t myFlag;

void task1(void)
{
  // do something
  while (!myFlag)
    ;
  // do something else
}

void task2(void)
{
  // do something
  myFlag = 1;
  // do something else
}

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

> --- In AVR-Chat@yahoogroups.com, Bob Paddock <bob.paddock@...> wrote:

> > Volatile is never the solution to a problem with threads.
> Nonsense. The volatile attribute is precisely the solution for *any* situation where a variable can be changed by an outside actor - even if it is another task.

No. http://blog.regehr.org/archives/28 "Nine ways to break your
systems code using volatile"  See #8.

Volatile means that all reads and writes happen to an address, even if
the compiler thinks they are not needed.  This has nothing to do with
concurrent access, which you have pointed out.

 > As I noted, volatile doesn't address atomicity of access nor (as
you alluded to) does it address serialization of resource access.
Those are different problems with different solutions.

Correct.

I believe that I have considered these issues in enough depth to weigh in here and try to sort out this dispute.  We are talking here about the use of the volatile modifier, so I have changed the discussion subject accordingly.

Don Kinzer is correct to write that "The volatile attribute must be used on any ... variables modified by another thread/task/process that may run at any time" (with the implicit assumption that you also want to access them outside that thread).  We could really roll up Don's examples (which he did not, by the way, claim to be exhaustive) into:
- hardware registers, and
- variables modified and inspected in different threads of execution
if we define "threads of execution" to include the main thread, interrupts, exceptions, pre-emptive threads running on the same machine and all threads running on different machines that can access the same storage.

Bob Paddock is incorrect to assert that "Volatile is never the solution to a problem with threads", whether by "problem" we are talking about a normal design challenge or something that has gone wrong.  The use of volatile variables to communicate between threads is a perfectly valid design technique and, if your threads are misbehaving, then the lack of a volatile qualifier somewhere should definitely be checked.

Calling on references for this discussion does not work because John Regehr and Arch Robison are also wrong (although Arch may have been taken out of context).  Volatile is *extremely useful* for multi-threaded programming.  It is idiotic to state otherwise.  Don generously writes "We may be arguing different points", so what could Bob (with support from an Associate Professor of Computer Science at the University of Utah and the architect of Threading Building Blocks at Intel) be arguing about?  He would be correct to argue that the use of the volatile qualifier is not always a sufficient technique to ensure the orderly passage of data between threads.  You also need to ensure atomicity and to worry about thread synchronization.  But Don has already mentioned this.  Possibly, Bob is trying to say that the simple use of the volatile qualifier, without explicit execution barriers and handling of atomicity, is *never* a workable solution (this appears to be John Regehr's conjecture).  In this he would also be incorrect.

The definition of the volatile qualifier in the C language specification is in terms of the "virtual machine" and "sequence points" and how the implementation of a compiler may, for reasons of efficiency, deviate from the virtual machine as long as the result is "the same".  This is all incomprehensible to most of us, so we work with folk-lore definitions, such as the one provided by Don, and get into arguments like this one (and worse).  The only thing missing from Don's treatment of the subject is the importance of the volatile qualifier when modifying a storage location (Don only writes about using a variable that can be modified elsewhere).  It is good practice to make sure that any variable you write to that can be read elsewhere is declared volatile.  Not doing this is far less likely to cause a problem, as the compiler is unlikely to optimize away a write, but the modifier serves as useful documentation.  It may also be essential when dealing with a hardware register that is locally defined.  If the compiler can determine that the definition of the variable is invisible outside of the translation unit and it cannot see any reads of the variable, it may remove writes, thus eliminating the hardware side-effect you are trying to achieve.  So, Don's rule is a good one.  Use volatile with hardware registers and variables modified and inspected in different threads, with a suitably broad definition of threads.

I will now give an example of passing data between threads of execution using *only* the volatile qualifier, without synchronization primitives.  Data is queued in a buffer supported by read and write access methods and also by unsigned read and write counters.  The counters must be at least as large as the number of entries in the buffer and must be of a type such that access by the source and sink of data is inherently atomic (for example, they are one memory word).  The data source is allowed to insert data into the buffer using the write access method and may modify the write count but may only examine the read count.  To find out whether there is space in the buffer for new data, it takes the unsigned difference between the write and read counts (modulo the count size) and if the result is smaller than the buffer size by the data size, it goes ahead and inserts the data and updates the write count (using unsigned arithmetic, ignoring overflow).  The data sink is allowed to remove data from the buffer using the read access method and may modify the read count but may only examine the write count.  To find out whether there is data in the buffer, it takes the unsigned difference between the read and write counts and if the result is non-zero, it goes ahead and removes the data and updates the read count.  Note that the buffer access methods and their control variables are not shared; in fact source and sink have no idea how the other actually accesses the data.  The only information shared between the threads are the counts.  Only the source is permitted to modify the write count and only the sink can modify the read count.  If one reads the count written by the other at around the time it is being written, it will either get the value before the write, in which case it won't see the new data this time around, or it will get the value after the write, in which case it will see the new data.  Both are valid outcomes.  The integrity of the data passing is ensured by good algorithm design, not by indifferent algorithm design and fancy thread synchronization.

My operating system (ECROS) uses precisely the above method in the implementation of buffered serial communication.  The buffers are circular buffers and the access methods are read and write indexes into the buffer (which can actually be implemented by taking the read and write counts modulo the buffer size).  For a more complicated buffer, such as a linked list, it is important to remember that the read and write access methods must not share any control variables and must not modify a buffer entry that they don't "own" (are not, respectively, in the process of removing or adding).

Graham Davies
ECROS Technology
www.ecrostech.com

--- In AVR-Chat@yahoogroups.com, "bayramdavies" <Yahoo37849@...> wrote:
> [...] if we define "threads of execution" to include the main thread,
> interrupts, exceptions, pre-emptive threads running on the same
> machine and all threads running on different machines that can
> access the same storage.
One other situation that must be included is when storage is modified by DMA.  While this could probably be subsumed in "threads running on different machines" it probably deserves explicit mention.

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

> Bob Paddock is incorrect to assert that "Volatile is never the solution to a problem with threads", whether by "problem" we are talking about a normal design challenge or something that has gone wrong. The use of volatile variables to communicate between threads is a perfectly valid design technique and, if your threads are misbehaving, then the lack of a volatile qualifier somewhere should definitely be checked.

I stand by what I said.  Volatile always needs to be used
appropriately, threads or no threads.

> [Bob] would be correct to argue that the use of the volatile qualifier is not always a sufficient technique to ensure the orderly passage of data between threads.

Exactly.

> You also need to ensure atomicity and to worry about thread synchronization. But Don has already mentioned this. Possibly, Bob is trying to say that the simple use of the volatile qualifier, without explicit execution barriers >and handling of atomicity, is *never* a workable solution (this appears to be John Regehr's conjecture). In this he would also be incorrect.

Obviously I fail to see how John and company are wrong?  We may be
having problems in our terminology.
A ISR()  changing a value in the background of a foreground task, to
me is not a 'thread'.

> The definition of the volatile qualifier in the C language specification is in terms of the "virtual machine" and "sequence points" and how the implementation of a compiler may, for reasons of efficiency, deviate from the virtual machine as long as the result is "the same". This is all incomprehensible to most of us,

ISO/IEC 9899:201x
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf

6.7.3 Type qualifiers: 134) A volatile declaration may be used to
describe an object corresponding to a memory-mapped
input/output port or an object accessed by an asynchronously
interrupting function. Actions on
objects so declared shall not be ''optimized out'' by an
implementation or reordered except as
permitted by the rules for evaluating expressions.

Search that document for 'volatile'.  There are places where applying
'volatile' blindly will produce code that is broken.

There is more than just adding 'volatile' on a variable or register to
make bug free code.
Issues such as:

* Locking / or Lock Free Atomic Access.
[Pure message passing like in Erlang is one way around locks/atomic
access, in that *nothing* is shared.]

* Code Motion

> It may also be essential when dealing with a hardware register that is locally defined. If the compiler can determine that the definition of the variable is invisible outside of the translation unit and it cannot see any reads of the >variable, it may remove writes, thus eliminating the hardware side-effect you are trying to achieve. So, Don's rule is a good one. Use volatile with hardware registers and variables modified and inspected in different threads, with >a suitably broad definition of threads.

So the argument is to blindly apply volatile?  Which leads to slow and
bloated code size.
I do not think any of us buy into that.

6.5.16 Assignment operators 3 (111): The *mplementation* is permitted
to read the object to determine the value but is not required to, even
when the object has volatile-qualified type.

Lets hope our AVR *implementation* is correct for Embedded use...

The standard points out several places that where volatile is
redundant or does not do what one expects from its usage.

> I will now give an example of passing data between threads of execution using *only* the volatile qualifier, without synchronization primitives.

Herb Sutter covers exactly this example in a multi-part series
starting in the September issue of Dr Dobb's journal:

Lock-Free Code: A False Sense of Security
http://www.drdobbs.com/cpp/210600279

> Data is queued in a buffer supported by read and write access methods and also by unsigned read and write counters. The counters must be at least as large as the number of entries in the buffer and must be of a type such >that access by the source and sink of data is inherently atomic (for example, they are one memory word).

No, it must be *explicitly* atomic, unless the counters are uint8_t on
our 8-Bit AVR's.

As we are discussing the AVR here, which is an 8-bit part, where the C
standard requires Int to be at lease 16 bits (a word), simply
declaring a 16 bit counter variable 'volatile', that is written inside
an ISR, and read outside of a ISR is now a bug,  if 'volatile' is the
sole thing being used.  My example shows exactly this problem, and its
solutions.  The problem shows up when the lower 8-bit value causes a
change to the upper 8-bit value (carry or borrow depending on the
implementation), when the ISR interrupts the read, and the ISR changes
the value that is being read.  There are two possible outcomes, one
that *may* be of no consequence (depends on the implementation), and
one that has the counter value off by a factor of 256.

> The integrity of the data passing is ensured by good algorithm design, not by indifferent algorithm design and fancy thread synchronization.

The implementation of the algorithm is of paramount importance.  A
good algorithm implemented badly, as Herb shows, produces buggy
products.

> The buffers are circular buffers and the access methods are read and write indexes into the buffer

A circular buffer that is limited to the size of the natural width of
the processor (8-bit for the AVR)  in question, that does not use
counters is the classic Head/Tail, may be found here:

http://websvn.hylands.org/filedetails.php?repname=Projects&path=%2Fcommon%2FCBUF.h&sc=1

On an 8-bit AVR trying to use a buffer larger than 256 once again
brings back the race condition that using volatile does not solve.


--
http://blog.softwaresafety.net/
http://www.designer-iii.com/
http://www.wearablesmartsensors.com/

Group,

In my opinion still, Don Kinzer's posts on where to use the volatile qualifier are unassailable and Bob Paddock's disagreement, "Volatile is never the solution to a problem with threads", is without merit.  Read on if you have the stomach for it.

Bob,

You write "I stand by what I said." and then write "Volatile always needs to be used appropriately, threads or no threads.".  This is not what you previously wrote.  If this is what you wish you had wrote, we're done here.

You write "I fail to see how John [Regehr] and company are wrong".  They are wrong by counter-example.  They propose that "volatile is almost useless for multi-threaded programming".  It isn't hard to be successful in multi-threaded programming design problems using only volatile and, to make the point, I provided an example.

You write "A ISR() changing a value ... to me is not a 'thread'".  I don't know why you bring this up.  The distinction is arbitrary and does not help the argument.  If you have a mechanism to safely pass data to and from interrupt threads, then you can use it between other types of execution threads as well, including multi-processor and DMA.

You quote ISO/IEC 9899:201x.  Footnote 134) says one of the things that Don says and does not support your argument.  You offer no support for your *new* argument that "There are places where applying 'volatile' blindly will produce code that is broken", although, depending on your definition of "broken", I could agree with you or not.  Again, this has no relevance to your previous argument that "Volatile is never the solution to a problem with threads".

You write "There is more than just adding 'volatile' on a variable or register to make bug free code".  Of course.  Nobody has said otherwise.  You must also design good algorithms, express them correctly in your chosen language, have a reliable compiler, etc, etc.

You write "So the argument is to blindly apply volatile?".  Now your post is looking like an attack, because this is certainly not the argument.  Don has written that the "volatile attribute must be used" on certain variables and goes on to explain how to figure out whether a variable should be volatile, with examples.  So the argument is precisely not what you claim it is, rather it is to apply volatile with your eyes open.

You quote ISO/IEC 9899:201x again.  Footnote 111 is taken entirely out of context.  In the paragraph that references the footnote, the document states that an assignment expression has a value (which, as we know, can be used as an rvalue in another assignment, or as a function argument, etc.).  It says that the value of the assignment expression is what has been stored in the object referenced by the left operand, which is kind-of obvious, but has to be said.  The footnote merely says that if the compiler needs the expression value, it can either read back the value of that object or just remember what it stored there and carry on.  (I would argue that if the object is volatile, the specification should pick one or the other and not leave this open as the read-back could have side-effects.  This is an area where the incomplete specification of the language can trip you up, but is easily avoided by not using the values of assignment expressions when the object referenced by the left operand is volatile.)

With regard to my "example of passing data between threads of execution using *only* the volatile qualifier", you write "Herb Sutter covers exactly this example in ...".  Well, this is a demonstrably false statement.  If you read only as far as page 2 of the article, you can see that the only thing that my example and the subject of the article have in common is the goal of passing data between producer and consumer threads.  You seem again to be more interested in discrediting me that supporting your argument.

In discussing my example, you write that counter access "must be *explicitly* atomic, unless the counters are uint8_t on our 8-Bit AVR's".  Well, on an 8-bit machine, they would naturally be 8 bits because that is the machine word size, so access to an 8-bit object is inherently atomic, as I required, and your comment is not helpful.  You describe the use of integer counters, the fact that the C language requires an integer to be at least 16 bits and the consequence that access to a 16-bit counter is not inherently atomic on an 8-bit machine and so the technique I have described will not work.  You are correct, but since I never said that it would work unless "counters ... [are] of a type such that access ... is inherently atomic" you again give the appearance of disagreeing with me without actually engaging my argument.

I'm done here, so, Bob, you may have the last word.

Graham Davies
ECROS Technology
www.ecrostech.com

Group,

While composing my previous post in reply to Bob Paddock, I read the Dr. Dobb's Journal article he referenced only as far as was necessary to determine that it was not about my algorithm (which demonstrates passing data between threads using only volatile control variables and without locks).  I have now read it in its entirety.  Although it is hard going, I think I understand it.

Herb Sutter's criticisms of the code he dissects center on the control variables used to keep track of the position of the data item due for next removal and the data item most recently inserted into the data queue.  He shows that they are of a type that does not guarantee atomic access.  He further states that the compiler is free to re-order object access around these types.  Both are fatal problems.

Interestingly, these are exactly the two problems I explicitly avoid in my algorithm.  If you re-read my description, you will see that I require access to the control variables (read and write counters) to be "inherently atomic", which in effect means that they are each a single machine word, 8 bits in the case of an AVR.  Second, I follow Don Kinzer's advice by declaring them volatile.  The compiler is not allowed to re-order object access on either side of an access to a volatile variable.

This article is, therefore, pretty much a direct counter-example to Bob's assertion that "Volatile is never the solution to a problem with threads".  Half of the problem exposed in the article is precisely that the control variables do not have the quality that the volatile qualifier would give them.

Graham Davies
ECROS Technology
www.ecrostech.com

As I understand it, Volatile variables are used when the variable is 
expected to be changed in Interrupt routines. Otherwise most global 
variables are by nature changeable at any time. But, interrupt routines 
do not normally allow outside variables to be changed unless they are 
volatile. Otherwise pointers must be used.

Further, volatile variables need to be declared in your global section.

i suggest this link efundies.com

rep

On 4/16/2012 6:33 AM, bayramdavies wrote:
>
> Group,
>
> In my opinion still, Don Kinzer's posts on where to use the volatile 
> qualifier are unassailable and Bob Paddock's disagreement, "Volatile 
> is never the solution to a problem with threads", is without merit. 
> Read on if you have the stomach for it.
>
> Bob,
>
> You write "I stand by what I said." and then write "Volatile always 
> needs to be used appropriately, threads or no threads.". This is not 
> what you previously wrote. If this is what you wish you had wrote, 
> we're done here.
>
> You write "I fail to see how John [Regehr] and company are wrong". 
> They are wrong by counter-example. They propose that "volatile is 
> almost useless for multi-threaded programming". It isn't hard to be 
> successful in multi-threaded programming design problems using only 
> volatile and, to make the point, I provided an example.
>
> You write "A ISR() changing a value ... to me is not a 'thread'". I 
> don't know why you bring this up. The distinction is arbitrary and 
> does not help the argument. If you have a mechanism to safely pass 
> data to and from interrupt threads, then you can use it between other 
> types of execution threads as well, including multi-processor and DMA.
>
> You quote ISO/IEC 9899:201x. Footnote 134) says one of the things that 
> Don says and does not support your argument. You offer no support for 
> your *new* argument that "There are places where applying 'volatile' 
> blindly will produce code that is broken", although, depending on your 
> definition of "broken", I could agree with you or not. Again, this has 
> no relevance to your previous argument that "Volatile is never the 
> solution to a problem with threads".
>
> You write "There is more than just adding 'volatile' on a variable or 
> register to make bug free code". Of course. Nobody has said otherwise. 
> You must also design good algorithms, express them correctly in your 
> chosen language, have a reliable compiler, etc, etc.
>
> You write "So the argument is to blindly apply volatile?". Now your 
> post is looking like an attack, because this is certainly not the 
> argument. Don has written that the "volatile attribute must be used" 
> on certain variables and goes on to explain how to figure out whether 
> a variable should be volatile, with examples. So the argument is 
> precisely not what you claim it is, rather it is to apply volatile 
> with your eyes open.
>
> You quote ISO/IEC 9899:201x again. Footnote 111 is taken entirely out 
> of context. In the paragraph that references the footnote, the 
> document states that an assignment expression has a value (which, as 
> we know, can be used as an rvalue in another assignment, or as a 
> function argument, etc.). It says that the value of the assignment 
> expression is what has been stored in the object referenced by the 
> left operand, which is kind-of obvious, but has to be said. The 
> footnote merely says that if the compiler needs the expression value, 
> it can either read back the value of that object or just remember what 
> it stored there and carry on. (I would argue that if the object is 
> volatile, the specification should pick one or the other and not leave 
> this open as the read-back could have side-effects. This is an area 
> where the incomplete specification of the language can trip you up, 
> but is easily avoided by not using the values of assignment 
> expressions when the object referenced by the left operand is volatile.)
>
> With regard to my "example of passing data between threads of 
> execution using *only* the volatile qualifier", you write "Herb Sutter 
> covers exactly this example in ...". Well, this is a demonstrably 
> false statement. If you read only as far as page 2 of the article, you 
> can see that the only thing that my example and the subject of the 
> article have in common is the goal of passing data between producer 
> and consumer threads. You seem again to be more interested in 
> discrediting me that supporting your argument.
>
> In discussing my example, you write that counter access "must be 
> *explicitly* atomic, unless the counters are uint8_t on our 8-Bit 
> AVR's". Well, on an 8-bit machine, they would naturally be 8 bits 
> because that is the machine word size, so access to an 8-bit object is 
> inherently atomic, as I required, and your comment is not helpful. You 
> describe the use of integer counters, the fact that the C language 
> requires an integer to be at least 16 bits and the consequence that 
> access to a 16-bit counter is not inherently atomic on an 8-bit 
> machine and so the technique I have described will not work. You are 
> correct, but since I never said that it would work unless "counters 
> ... [are] of a type such that access ... is inherently atomic" you 
> again give the appearance of disagreeing with me without actually 
> engaging my argument.
>
> I'm done here, so, Bob, you may have the last word.
>
> Graham Davies
> ECROS Technology
> www.ecrostech.com
>
>

R E Purcella wrote:

> As I understand it, Volatile variables are
> used when the variable is expected to be
> changed in Interrupt routines.

That is one situation.  See Don Kinzer's post of Apr 13 for others.

> Otherwise most global variables are by
> nature changeable at any time.

No, this is not true.  If a "global" variable (one with external linkage) is not declared volatile, then the compiler will assume that it cannot change during the execution of the code being compiled other than by an assignment it can see within that code.

> But, interrupt routines do not normally allow
> outside variables to be changed unless they are 
> volatile. Otherwise pointers must be used.

I can't make any sense of this.  You can do whatever you want in an interrupt service routine, there is no issue of being "not allowed" to do certain things.

> Further, volatile variables need to be declared
> in your global section.

No, you can write the declaration of a global variable anywhere you want to, according to your style and needs.
 
> i suggest this link efundies.com

I suggest that for complex issues such as this you avoid sites that describe themselves as "a collection of fun, free, electronics projects and advice" and instead stick to places visited by a good number of professionals who are likely to know what they are talking about.

Graham Davies
ECROS Technology
www.ecrostech.com

--- In AVR-Chat@yahoogroups.com, "bayramdavies" <Yahoo37849@...> wrote:
> > Otherwise most global variables are by
> > nature changeable at any time.
> No, this is not true.  If a "global" variable (one with external
> linkage) is not declared volatile, then the compiler will assume
> that it cannot change during the execution of the code being
> compiled other than by an assignment it can see within that code.

Note, however, that if there are any calls to other functions in the code region of interest the compiler must assume that any particular global variable has been modified unless it can determine conclusively otherwise.

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

Don Kinzer wrote:

>... if there are any calls to other functions in
> the code region of interest the compiler must
> assume that any particular global variable
> has been modified ...

Good point.

> ... unless it can determine conclusively otherwise.

Which, I think, it generally can't for the C language.

What I was getting at, as I'm sure you realized, is the common case of busy-waiting on a change in a global variable.  Without the volatile modifier, the compiler will see no reason to repeatedly read the value as, while executing the busy-wait loop, it is impossible for the code that it is compiling to modify the value, either by a direct assignment or by an external function call.  The volatile qualifier "tells" the compiler that "something else" might modify it while the loop is running.

In other news, I am getting so fed up with this issue cropping up several times a year on various discussion groups that I am trying to prepare a definitive statement on when you should use the volatile qualifier and exactly what effect it has.  I am hoping that, if I can get the appropriate peer reviews, people will reference this when the subject first arises.  This might avoid the posting of yet more miss-information (which, as you point out, I am contributing to myself).

Graham Davies
ECROS Technology
www.ecrostech.com

--- In AVR-Chat@yahoogroups.com, "bayramdavies" <Yahoo37849@...> wrote:
>> ... unless it can determine conclusively otherwise.
> Which, I think, it generally can't for the C language.
For most languages, C included, it is conceptually easy: look at a function and see what global variables it might modify and then do the same for any function that it might call and so on.  Different compilers may go to varying lengths to try to optimize out reloading of globals and there are situations that complicate matters (e.g. writing through a pointer having an unknown value).  

If you compile the example code below with WinAVR_20100110 using -Os you'll see that the compiler inlines the foo() code into main() and determines that the variable b1 is loop invariant.  Granted, it is a simple example but it shows that it can be and is done.

uint8_t b1;
uint8_t b2;

void foo(void)
{
    b2++;
}

int main(void)
{
    while (b1)
        foo();
    for (;;)
        ;
}


Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

In AVR-Chat@yahoogroups.com, Don Kinzer wrote:

> For most languages, C included, it is conceptually
> easy [to determine whether or not a function modifies
> a global variable]: look at a function and see what
> global variables it might modify and then do the same
> for any function that it might call and so on.

But, doesn't a C compiler, by the very design of the language, limit itself to the consideration of a single translation unit at a time?  That would mean that if a called function is in a different unit there is no way of knowing what global variables are touched.  And, if the variable *isn't* modified by a function in a different unit, why would it be "global" in the first place?

> ... there are situations that complicate matters (e.g.
> writing through a pointer having an unknown value).  

So, even if the function is in the same unit, if it writes through a pointer the compiler again has to throw up its hands.  (This applies to file-scope variables, too.)

> If you compile the example code below ...

I wonder if the inlining is done first, in which case the optimization is unremarkable.

Graham Davies
ECROS Technology
www.ecrostech.com

--- In AVR-Chat@yahoogroups.com, "bayramdavies" <Yahoo37849@...> wrote:
> But, doesn't a C compiler [...] limit itself to the consideration
> of a single translation unit at a time?  That would mean that if
> a called function is in a different unit there is no way of
> knowing what global variables are touched.
I suspect that that is usually the case.  The gcc compiler has an option to compile all of the files at one go (used for whole program optimization).  With that option, the compiler *could* examine functions defined in another file.  I don't know if it does or not.

>And, if the variable *isn't* modified by a function in a different
>unit, why would it be "global" in the first place?
A global variable needn't be modified by *every* function in the system.  The optimization opportunity we're discussing only depends on whether or not a particular set of functions modifies a particular global variable.

> I wonder if the inlining is done first, in which case the 
>optimization is unremarkable.
Possibly.  I didn't experiment with options to suppress the inlining.  My goal was to show that in at least one case the compiler deduced that a global variable was loop invariant in spite of the presence of a call to another function in the loop.

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

I found this statement very helpful.

"No, this is not true. If a "global" variable (one with external 
linkage) is not declared volatile, then the compiler will assume that it 
cannot change during the execution of the code being compiled other than 
by an assignment it can see within that code."

Thank you for verifying what i was trying to say. This is the problem, 
and the solution. An ISR will change volatile variables. In my 
experience and from reading the winavr manual, ISR routines will not 
change non-volatile variables. That is to say, non-volatile variables 
will not change on the outside of the ISR function; with or without an 
assignment. the variable must be volatile.

But, in other functions I can change any variable simply by giving it a 
new value by assignment. So long as it is global to my function.

(simple explanation can be found at efundies.com not my site fun site or 
not)


Perhaps, global means something different between us? I use global to 
mean variables in my code declared before MAIN, declared outside of 
other functions, that are changeable/changed during execution. It is 
understood that variables are changed by assignment. and without an 
assignment they will not be changed. Generally code can assign a new 
value to a variable at any time. Except as you say, when a linkage is 
needed.

My word "otherwise" draws a contrast between volatile variables and 
other variables that are not volatile.



rep

On 4/17/2012 5:06 AM, bayramdavies wrote:
>
> R E Purcella wrote:
>
> > As I understand it, Volatile variables are
> > used when the variable is expected to be
> > changed in Interrupt routines.
>
> That is one situation. See Don Kinzer's post of Apr 13 for others.
>
> > Otherwise most global variables are by
> > nature changeable at any time.
>
>
>
> > But, interrupt routines do not normally allow
> > outside variables to be changed unless they are
> > volatile. Otherwise pointers must be used.
>
> I can't make any sense of this. You can do whatever you want in an 
> interrupt service routine, there is no issue of being "not allowed" to 
> do certain things.
>
> > Further, volatile variables need to be declared
> > in your global section.
>
> No, you can write the declaration of a global variable anywhere you 
> want to, according to your style and needs.
>
> > i suggest this link efundies.com
>
> I suggest that for complex issues such as this you avoid sites that 
> describe themselves as "a collection of fun, free, electronics 
> projects and advice" and instead stick to places visited by a good 
> number of professionals who are likely to know what they are talking 
> about.
>
> Graham Davies
> ECROS Technology
> www.ecrostech.com
>
>

R E Purcella wrote (although not in this order):

> Perhaps, global means something different between us?
> I use global to mean variables in my code declared
> before MAIN, declared outside of other functions

OK, let's straighten this out.  "Global" means in scope (and thus able to be accessed) everywhere in a program.  There is, strictly speaking, no global scope in the C language.  Variables declared
outside of a function have "file scope".  If they have the "static"
storage class, they are given internal linkage and so they cannot be referenced outside of the unit in which they are defined (the linker will error).  If they don't have static, they have external linkage and can be accessed in other units.  So, to produce the effect of a global variable, declare the variable in a header file that is included in all source files, don't use static, and define the variable in the source file of your choice.  Many (most?) professionals will avoid this whenever possible so as to limit the "connectedness" of the program.  The more global variables you have, the more difficult it becomes to reason about a program and maintain its correctness.

> An ISR will change volatile variables. In my 
> experience and from reading the winavr manual,
> ISR routines will not change non-volatile
> variables. That is to say, non-volatile variables 
> will not change on the outside of the ISR
> function; with or without an assignment.
> the variable must be volatile.

This is so wrong that I am forced to assume I misunderstand you.  There is nothing in the C language, the compiler or the hardware that prevents an ISR from changing non-volatile variables.  If this is really what you are saying, I suggest that you completely discard your present understanding of volatile and re-read this thread.

> But, in other functions I can change any variable
> simply by giving it a new value by assignment.
> So long as it is global to my function.

You can access a variable whenever it is in scope, whether that is local or file scope.  The compiler will allow this.  The problem we are discussing is how the accesses that you have written in your source code map to the generated machine code.

> (simple explanation can be found at efundies.com ...)

Again, you would do better to hang with professionals than fun-seekers for complex issues such as this.  The only reference to volatile on the efundies.com Web site that I can find using Google is a brief example of where you should use it, which Don covered in this thread days ago, followed by a fantastically garbled explanation of why you need to do this.  If this is where you got your understanding of volatile, again, I suggest you start again.  An explanation is not necessarily good because it is simple.  Sometimes you just have to suck it up and wade into the complexity if you want to properly understand something.

Graham Davies
ECROS Technology
www.ecrostech.com

To be fair, the avrlibc does give the misleading impression that volatile is required for a variable that is only updated in an ISR routine and not in the "main" loop of the code with a very limited example but it is response to a faq.
 
This thread started as a fairly simple question although the complete answer is complex the actually answer to the initial query was not so.
 
Confusion has certainly defeated clarity.
 
Steven Holder
Scope Communications.
www.scope-uk.com
 


________________________________

From: bayramdavies <Yahoo37849@ecrostech.com>
To: AVR-Chat@yahoogroups.com 
Sent: Wednesday, 18 April 2012, 13:42
Subject: [AVR-Chat] Re: Volatile modifier


  
R E Purcella wrote (although not in this order):

> Perhaps, global means something different between us?
> I use global to mean variables in my code declared
> before MAIN, declared outside of other functions

OK, let's straighten this out. "Global" means in scope (and thus able to be accessed) everywhere in a program. There is, strictly speaking, no global scope in the C language. Variables declared
outside of a function have "file scope". If they have the "static"
storage class, they are given internal linkage and so they cannot be referenced outside of the unit in which they are defined (the linker will error). If they don't have static, they have external linkage and can be accessed in other units. So, to produce the effect of a global variable, declare the variable in a header file that is included in all source files, don't use static, and define the variable in the source file of your choice. Many (most?) professionals will avoid this whenever possible so as to limit the "connectedness" of the program. The more global variables you have, the more difficult it becomes to reason about a program and maintain its correctness.

> An ISR will change volatile variables. In my 
> experience and from reading the winavr manual,
> ISR routines will not change non-volatile
> variables. That is to say, non-volatile variables 
> will not change on the outside of the ISR
> function; with or without an assignment.
> the variable must be volatile.

This is so wrong that I am forced to assume I misunderstand you. There is nothing in the C language, the compiler or the hardware that prevents an ISR from changing non-volatile variables. If this is really what you are saying, I suggest that you completely discard your present understanding of volatile and re-read this thread.

> But, in other functions I can change any variable
> simply by giving it a new value by assignment.
> So long as it is global to my function.

You can access a variable whenever it is in scope, whether that is local or file scope. The compiler will allow this. The problem we are discussing is how the accesses that you have written in your source code map to the generated machine code.

> (simple explanation can be found at efundies.com ...)

Again, you would do better to hang with professionals than fun-seekers for complex issues such as this. The only reference to volatile on the efundies.com Web site that I can find using Google is a brief example of where you should use it, which Don covered in this thread days ago, followed by a fantastically garbled explanation of why you need to do this. If this is where you got your understanding of volatile, again, I suggest you start again. An explanation is not necessarily good because it is simple. Sometimes you just have to suck it up and wade into the complexity if you want to properly understand something.

Graham Davies
ECROS Technology
www.ecrostech.com




[Non-text portions of this message have been removed]

STEVEN HOLDER wrote:

> Confusion has certainly defeated clarity.

Do you have a suggestion on how we can achieve clarity?

Graham.

--- In AVR-Chat@yahoogroups.com, "bayramdavies" <Yahoo37849@...> wrote:
>Do you have a suggestion on how we can achieve clarity?

I think this statement describes the issue:

If while examining a particular function or smaller code sequence it is determined that (a) the content of a storage location read by that code sequence might be changed by a means that cannot be deduced, or (b) the content of a storage location modified by that code sequence might be read by a means that that cannot be deduced, or (c) the act of reading or writing a storage location might produce side effects, then the storage declaration must have the volatile modifier.

Examples of (a) include:
- storage modified by an ISR
- storage modified by another task/thread/process
- storage in multi-port memory accessed by an external writer
- storage modified by DMA
- storage represented by hardware registers

Examples of (b) include:
- storage read by an ISR
- storage read by another task/thread/process
- storage in multi-port memory accessed by an external reader

Examples of (c) include:
- storage represented by hardware registers

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

Don Kinzer wrote:

> If while examining a particular function
> or smaller code sequence it is determined
> that (a) the content of ... [etc., etc.]

Well, that looks good to me, but then I've been agreeing with you all along.  What I'm trying to get at, though, is why people are still debating this and is there something that could be said that would do it for everyone.  There are words that you use, Don, in particular "deduced", where I fill in what I think is your meaning, and thus I agree with your explanation.  But, if someone looks at "deduced" and can't figure out what you mean or gets it wrong, then I can see how the job is not yet done.

I am still grappling with my "definitive" description of volatile, which I hope to submit for review, but there is also work to do for which I actually hope to get paid, so it's not ready yet.

Graham.

On 4/18/2012 5:18 PM, bayramdavies wrote:
> I am still grappling with my "definitive" description of volatile,
> which I hope to submit for review, but there is also work to do for
> which I actually hope to get paid, so it's not ready yet.

How about

-----
volatile controls access to a variable.  Reads and writes to a volatile 
variable must be performed by the compiler in the order written and 
exactly as many times as written.
-----

I think that's accurate, complete and deceptively simple.  All our other 
understandings of its use come from that definition.  A formal 
definition would include more formal definitions but this is a little 
simpler to read if not quite as accurate. It does ignore sequence points 
and reordering within an expression, but if you are concerned about the 
latter you are probably making your code too complex.

Note that this means that

const volatile int *f;

is not only syntactically correct but useful.  And yes, I have used it.

Robert

Oh my found an on-line version of cdecl http://cdecl.org/

Try

explain const volatile int *f

if you have trouble parsing the above declaration


-- 
  From the Divided by a Common Language File (Edited to protect the guilty)
ME - "I'd like to get Price and delivery for connector Part # XXXXX"
Dist./Rep - "$X.XX Lead time 37 days"
ME - "Anything we can do about lead time?  37 days seems a bit high."
Dist./Rep - "that is the lead time given because our stock is live....
we currently have stock."

On 18/04/2012 23:44, Robert Adsett wrote:
>
> On 4/18/2012 5:18 PM, bayramdavies wrote:
> > I am still grappling with my "definitive" description of volatile,
> > which I hope to submit for review, but there is also work to do for
> > which I actually hope to get paid, so it's not ready yet.
>
> How about
>
> -----
> volatile controls access to a variable. Reads and writes to a volatile
> variable must be performed by the compiler in the order written and
> exactly as many times as written.
> -----
>
> I think that's accurate, complete and deceptively simple. All our other
> understandings of its use come from that definition. A formal
> definition would include more formal definitions but this is a little
> simpler to read if not quite as accurate. It does ignore sequence points
> and reordering within an expression, but if you are concerned about the
> latter you are probably making your code too complex.
>
> Note that this means that
>
> const volatile int *f;
>
> is not only syntactically correct but useful. And yes, I have used it.
>
> Robert
>
> Oh my found an on-line version of cdecl http://cdecl.org/
>
> Try
>
> explain const volatile int *f
>
> if you have trouble parsing the above declaration
>
> -- 
> From the Divided by a Common Language File (Edited to protect the guilty)
> ME - "I'd like to get Price and delivery for connector Part # XXXXX"
> Dist./Rep - "$X.XX Lead time 37 days"
> ME - "Anything we can do about lead time? 37 days seems a bit high."
> Dist./Rep - "that is the lead time given because our stock is live....
> we currently have stock."
>
> 
Robert,

Excellent.

Steven.

-- 
*Steven Holder*
s.holder123@btinternet.com
/www.sh-tec.co.uk/
//


[Non-text portions of this message have been removed]

> Do you have a suggestion on how we can achieve clarity?

I haven't been following every message in this list, so pardon me if I am repeating something.

It seems to me that the lack of clarity is because people are attempting to define the volatile storage class based on when and how to use it, not what it is.

If you define volatile by what it is, then its usage follows rather naturally.  The storage class volatile simply instructs the compile to avoid optimization of any operations involving the thing that is marked volatile.

The name volatile is itself misleading because it implies one usage of the storage class rather that what it is. nooptimize would be a more meaningful name.

[This, by the way, is a classic job interview C question.  "What does the volatile storage class mean?"  If you give any anser other than the above answer, you would probably fail the interview question.]

Now how does that tie into the rest of this discussion.  Well, the optimizer does things that you may not want it to do.  For example, if you have a load from a symbolic name within a loop but never modify the value within the loop, the optimizer is free to move the load outside of the loop.  This would be catastrophic if the thing you are loading is a register or is a value that could be modified by an interrupt or another thread.

volatile says don't optimize... don't move the load outside the loop.

Basically, anytime the compiler thinks it knows the value of something, it can suppress re-loading it as part of the optimization.

Another case is where the optimizer notices that a variable read but is never used, in that case the optimizer says, "the variable is never used I can eliminate it".  But if you are reading a machine register you probably want to read the value even though you don't use the value that you read.

volatile says don't optimize... read the value even though you don't use it.

Here is a complete unrelated use of volatile.  Suppose I have a timing loop like this:

void wait_a_bit(void)
{
  int i;
  for (i = 0; i < 1000; i++)
  ;
}

What can go wrong with that?  Well, first if you are doing really aggressive optimization, the compiler might decide that function does nothing and eliminate it completely.

But suppose it doesn't do that. That is still a problem... the problem is that the variable i will be a memory location if optimization is OFF but will be a register value if the optimization is ON.  So if I calibrated my timing loop with optimization OFF, it will be broken when optimization is turned on.  The fix?

volatile says don't optimize... don't convert the variable i to a register variable.

void wait_a_bit(void)
{
  volatile int i;
  for (i = 0; i < 1000; i++)
  ;
}

will behave approximately the same whether optimization is enabled or not.

volatile can help in many different ways and is easy to understand when to use it if you remember that all it does is turn optimization off.

Greg

Gregory N wrote (in part):

> It seems to me that the lack of clarity is
> because people are attempting to define the
> volatile storage class based on when and how
> to use it, not what it is.

OK, well, the above is a kind-of self-fulfilling observation, because volatile isn't a storage class.  I don't want to get combative about this, but my personal opinion is that the lack of clarity is caused by people shooting from the hip.

> If you define volatile by what it is, then
> its usage follows ...

I think that this would not work for the less advanced users.  If you sat a group of people in a classroom and told them in great detail what a car is, not all of them would be able to go outside and drive one.

> The name volatile is itself misleading ...

Indeed, as are other C language keywords, such as const and static.

> ... job interview C question.  "What does
> the volatile storage class mean?"

An almost impossible question to answer.  If you ignore the mistake in calling volatile a storage class, the interviewer can assume that you don't know what a storage class is.  If you point this out and it turns out that the interviewer didn't intend this mistake, he or she will feel threatened by you and not hire you.
 
> volatile says don't optimize... don't convert
> the variable i to a register variable.

I'm going to try a shot from the hip here myself and say that I don't think that this is correct.  A variable declared in a function without the static storage class specifier has the storage class auto.  This defines the lifetime of the variable as the execution of the smallest enclosing block and the starting value as unspecified.  It says nothing about where the compiler has to put the variable.  The compiler can put it in a register (if one is available), on the data stack (if there is one) or in ordinary memory.  This is not an optimization issue and has nothing to do with the volatile type qualifier.

Graham Davies
ECROS Technology
www.ecrostech.com

Having a "the complete reference C++" by Herbert Schildt, third edition, 
I tackled his definition of volatile. He gives it 2 paragraphs. His 
opening statement is: "The modifier volatile tells the compiler that a 
variable's value may be changed in ways not explicitly specified by the 
program."

The point being "optimization" sometimes needs to be curbed, or not 
performed, in order to keep the compiler from assuming the variable 
hasn't changed. Volatile modifies the variable and keeps the compiler 
from making optimization assumptions. As your example illustrates.

My friend says "If you are going to modify a global variable in your ISR 
**or you are going to modify it in your main loop and then use it in 
your ISR, you need to declare that variable as volatile**, to tell the 
compiler to not use the optimized value and assume that it hasn't changed."

So I understand him to be saying Volatile tells the compiler 1) not to 
use an optimized value 2) not to assume that the value hasn't changed

rep

On 4/19/2012 9:31 AM, Gregory N wrote:
>
> > Do you have a suggestion on how we can achieve clarity?
>
> I haven't been following every message in this list, so pardon me if I 
> am repeating something.
>
> It seems to me that the lack of clarity is because people are 
> attempting to define the volatile storage class based on when and how 
> to use it, not what it is.
>
> If you define volatile by what it is, then its usage follows rather 
> naturally. The storage class volatile simply instructs the compile to 
> avoid optimization of any operations involving the thing that is 
> marked volatile.
>
> The name volatile is itself misleading because it implies one usage of 
> the storage class rather that what it is. nooptimize would be a more 
> meaningful name.
>
> [This, by the way, is a classic job interview C question. "What does 
> the volatile storage class mean?" If you give any anser other than the 
> above answer, you would probably fail the interview question.]
>
> Now how does that tie into the rest of this discussion. Well, the 
> optimizer does things that you may not want it to do. For example, if 
> you have a load from a symbolic name within a loop but never modify 
> the value within the loop, the optimizer is free to move the load 
> outside of the loop. This would be catastrophic if the thing you are 
> loading is a register or is a value that could be modified by an 
> interrupt or another thread.
>
> volatile says don't optimize... don't move the load outside the loop.
>
> Basically, anytime the compiler thinks it knows the value of 
> something, it can suppress re-loading it as part of the optimization.
>
> Another case is where the optimizer notices that a variable read but 
> is never used, in that case the optimizer says, "the variable is never 
> used I can eliminate it". But if you are reading a machine register 
> you probably want to read the value even though you don't use the 
> value that you read.
>
> volatile says don't optimize... read the value even though you don't 
> use it.
>
> Here is a complete unrelated use of volatile. Suppose I have a timing 
> loop like this:
>
> void wait_a_bit(void)
> {
> int i;
> for (i = 0; i < 1000; i++)
> ;
> }
>
> What can go wrong with that? Well, first if you are doing really 
> aggressive optimization, the compiler might decide that function does 
> nothing and eliminate it completely.
>
> But suppose it doesn't do that. That is still a problem... the problem 
> is that the variable i will be a memory location if optimization is 
> OFF but will be a register value if the optimization is ON. So if I 
> calibrated my timing loop with optimization OFF, it will be broken 
> when optimization is turned on. The fix?
>
> volatile says don't optimize... don't convert the variable i to a 
> register variable.
>
> void wait_a_bit(void)
> {
> volatile int i;
> for (i = 0; i < 1000; i++)
> ;
> }
>
> will behave approximately the same whether optimization is enabled or not.
>
> volatile can help in many different ways and is easy to understand 
> when to use it if you remember that all it does is turn optimization off.
>
> Greg
>
>

R E Purcella wrote:

> ... "the complete reference C++" by Herbert
> Schildt, third edition ... [says] ... "The
> modifier volatile tells the compiler that a 
> variable's value may be changed in ways not
> explicitly specified by the program."

You see, this isn't actually correct.  I think the author is trying to make a compromise between telling us what the volatile type qualifier (not "modifier", there's the first mistake) actually is and when to use it.  Sentences involving what you "tell the compiler" are always going to be tricky.  If you're "telling" the compiler anything with the volatile qualifier, you're telling it that accesses to the qualified object (not variable, second mistake) must be done in strict conformity with the abstract machine used to define the semantics of the C language in the specification.  You're not telling the compiler why you're asking for this (third mistake) and the "value may be changed in ways not explicitly specified by the program" is only one of the reasons you might want strict conformance (fourth mistake).

> My friend says "... you need to declare that
> variable as volatile, to tell the compiler
> to not use the optimized value ..."

What the heck is "the optimized value"?  This is nonsense.  If you think you understand it, then that's because you think you already understand volatile and you're able to map this gobbledygook onto your understanding.

> So I understand him to be saying Volatile
> tells the compiler 1) not to use an optimized
> value 2) not to assume that the value hasn't
> changed

This is a poor description of volatile for reasons presented above.

I have yet to come across a description of volatile that conforms to what is written in the standard, is complete, correct and is in all situations helpful to people who do not fully understand the terminology of the standard, compilation and software engineering in general.  I am starting to think that nobody would read such a description as it would necessarily be quite long and people's preference is for short over complete and correct.

You might think, for example, that Keil (part of ARM) could get this right.  But, they gave the job of writing this description of the volatile type qualifier:

http://www.keil.com/support/man/docs/c51/c51_le_volatile.htm

to someone who has a pretty good understanding of the C language but no understanding of hardware.  Can you spot the glaring problem?

Graham Davies
ECROS Technology
www.ecrostech.com

--- In AVR-Chat@yahoogroups.com, R E Purcella <repurcella@...> wrote:
> Use volatile variables in ISR routines or 
> strange things happen.
That sure sounds like a superstition, somewhat akin to "don't walk under ladders or bad things will happen".  Neither statement suggests any understanding of an actual causal relationship.

You should know why it is necessary or not necessary in each particular case.  I would aver that an ISR that cannot be interrupted will work perfectly well without volatile (and if present it can be safely cast away) for normal variables located in single port RAM with no DMA activity.  If you don't know why that is true then you don't yet understand the issue.

Don Kinzer
ZBasic Microcontrollers
http://www.zbasic.net

I think you are nit picking words.

Optimized value was explained by Gregory N.

Many C, C++ code lines are directives to the compiler: include, define . 
. . Apparently volatile is as well.

I suppose that volatile is indeed far more complex in execution. But, 
hardly needed in order to type in "volatile int speed;" So long as I do 
understand that speed might be changed in unexpected ways by the 
compiler when it optimizes my code. I therefore protect my code, and 
variable, by saying VOLATILE. Use volatile variables in ISR routines or 
strange things happen.

I see it then as an optimization switch. Namely, optimization gets 
turned off for the code where a volatile variable occurs.

And that has been the gobblegook point of several writers all along.

rep

On 4/20/2012 7:04 AM, bayramdavies wrote:
>
>
>
> R E Purcella wrote:
>
> > ... "the complete reference C++" by Herbert
> > Schildt, third edition ... [says] ... "The
> > modifier volatile tells the compiler that a
> > variable's value may be changed in ways not
> > explicitly specified by the program."
>
> You see, this isn't actually correct. I think the author is trying to 
> make a compromise between telling us what the volatile type qualifier 
> (not "modifier", there's the first mistake) actually is and when to 
> use it. Sentences involving what you "tell the compiler" are always 
> going to be tricky. If you're "telling" the compiler anything with the 
> volatile qualifier, you're telling it that accesses to the qualified 
> object (not variable, second mistake) must be done in strict 
> conformity with the abstract machine used to define the semantics of 
> the C language in the specification. You're not telling the compiler 
> why you're asking for this (third mistake) and the "value may be 
> changed in ways not explicitly specified by the program" is only one 
> of the reasons you might want strict conformance (fourth mistake).
>
> > My friend says "... you need to declare that
> > variable as volatile, to tell the compiler
> > to not use the optimized value ..."
>
> What the heck is "the optimized value"? This is nonsense. If you think 
> you understand it, then that's because you think you already 
> understand volatile and you're able to map this gobbledygook onto your 
> understanding.
>
> > So I understand him to be saying Volatile
> > tells the compiler 1) not to use an optimized
> > value 2) not to assume that the value hasn't
> > changed
>
> This is a poor description of volatile for reasons presented above.
>
> I have yet to come across a description of volatile that conforms to 
> what is written in the standard, is complete, correct and is in all 
> situations helpful to people who do not fully understand the 
> terminology of the standard, compilation and software engineering in 
> general. I am starting to think that nobody would read such a 
> description as it would necessarily be quite long and people's 
> preference is for short over complete and correct.
>
> You might think, for example, that Keil (part of ARM) could get this 
> right. But, they gave the job of writing this description of the 
> volatile type qualifier:
>
> http://www.keil.com/support/man/docs/c51/c51_le_volatile.htm
>
> to someone who has a pretty good understanding of the C language but 
> no understanding of hardware. Can you spot the glaring problem?
>
> Graham Davies
> ECROS Technology
> www.ecrostech.com
>
>

On 4/20/2012 7:35 PM, Don Kinzer wrote:
> --- In AVR-Chat@yahoogroups.com, R E Purcella<repurcella@...>
> wrote:
>> Use volatile variables in ISR routines or strange things happen.
> That sure sounds like a superstition, somewhat akin to "don't walk
> under ladders or bad things will happen".  Neither statement suggests
> any understanding of an actual causal relationship.
>
> You should know why it is necessary or not necessary in each
> particular case.  I would aver that an ISR that cannot be interrupted
> will work perfectly well without volatile (and if present it can be
> safely cast away) for normal variables located in single port RAM
> with no DMA activity.  If you don't know why that is true then you
> don't yet understand the issue.

Yes, and there are cases where volatile is necessary even though the 
value cannot be read and will not change unexpectedly.

Robert
-- 

  From the Divided by a Common Language File (Edited to protect the guilty)
ME - "I'd like to get Price and delivery for connector Part # XXXXX"
Dist./Rep - "$X.XX Lead time 37 days"
ME - "Anything we can do about lead time?  37 days seems a bit high."
Dist./Rep - "that is the lead time given because our stock is live....
we currently have stock."

On 4/18/2012 7:00 PM, Steven Holder wrote:
> Robert,
>
> Excellent.

Thanks Steven, I prefer an explanation in terms of what is promised 
rather than how you would use it.

The discussion here is amply illustrating why.

Robert

-- 

  From the Divided by a Common Language File (Edited to protect the guilty)
ME - "I'd like to get Price and delivery for connector Part # XXXXX"
Dist./Rep - "$X.XX Lead time 37 days"
ME - "Anything we can do about lead time?  37 days seems a bit high."
Dist./Rep - "that is the lead time given because our stock is live....
we currently have stock."

I've been following this discussion with some amusement and even some interest at times.  :-)

Some folks have said that volatile is not a good keyword choice for what it instructs the compiler to do, but I think it is a great word, because it accurately describes the nature of the storage that it qualifies.  A quick look in the dictionary shows several definitions:

vol·a·tile   [vol-uh-tl, -til or, especially Brit., -tahyl]  Show IPA
adjective
1. evaporating rapidly; passing off readily in the form ofvapor: Acetone is a volatile solvent.
2. tending or threatening to break out into open violence;explosive: a volatile political situation.
3. changeable; mercurial; flighty: a volatile disposition.
4. (of prices, values, etc.) tending to fluctuate sharply andregularly: volatile market conditions.
5. fleeting; transient: volatile beauty.

Ignoring definitions 1 and 2, it's not a liquid, and it's not going to explode. :-)  But definitions 3, 4, and 5 are all relevant and applicable.

The essence of it is that it instructs the compiler that this "storage area", is, as the name implies, volatile.  Meaning, the content may change outside of the compiler's compile-time knowledge, or, in the case of some hardware ports/registers, accesses to said storage have side effects hardware related side-effects (clearing of certain bits after a read, and things like that).  Therefore, the compiler will not read a volatile storage area and save it in a register or other location for more rapid use later (an optimization).  It also must not change the order of execution of other operations before or after a volatile storage area is read or written.  Quite simply, a storage area declared with the volatile attribute, is volatile and can change "out of band", or have side-effects, and therefore it's value can never be saved or assumed by the compiler, it must always be accessed directly each and every time as directed by the programmer.

Therefore, I think the writers of the C standard chose a good name for the volatile type qualifier.  And it's usage is really pretty straightforward.  Yes, it sometimes trips up newcomers to embedded programming, low level OS programming, and so forth.  But that's just the learning curve that we all went through.  Hopefully when newcomers ask those questions, which usually start out saying that the compiler has a bug, haha, we are nice and can direct them to the FAQ, cut and paste the standard answer, etc.  We were all there at one time.  :-)

-Brian
-- 
Brian Dean
BDMICRO LLC
http://www.bdmicro.com/

On Apr 20, 2012, at 7:11 PM, R E Purcella wrote:

> I think you are nit picking words.
> 
> Optimized value was explained by Gregory N.
> 
> Many C, C++ code lines are directives to the compiler: include, define . 
> . . Apparently volatile is as well.
> 
> I suppose that volatile is indeed far more complex in execution. But, 
> hardly needed in order to type in "volatile int speed;" So long as I do 
> understand that speed might be changed in unexpected ways by the 
> compiler when it optimizes my code. I therefore protect my code, and 
> variable, by saying VOLATILE. Use volatile variables in ISR routines or 
> strange things happen.
> 
> I see it then as an optimization switch. Namely, optimization gets 
> turned off for the code where a volatile variable occurs.
> 
> And that has been the gobblegook point of several writers all along.
> 
> rep
> 
> 
> 
> On 4/20/2012 7:04 AM, bayramdavies wrote:
>> 
>> 
>> 
>> R E Purcella wrote:
>> 
>>> ... "the complete reference C++" by Herbert
>>> Schildt, third edition ... [says] ... "The
>>> modifier volatile tells the compiler that a
>>> variable's value may be changed in ways not
>>> explicitly specified by the program."
>> 
>> You see, this isn't actually correct. I think the author is trying to 
>> make a compromise between telling us what the volatile type qualifier 
>> (not "modifier", there's the first mistake) actually is and when to 
>> use it. Sentences involving what you "tell the compiler" are always 
>> going to be tricky. If you're "telling" the compiler anything with the 
>> volatile qualifier, you're telling it that accesses to the qualified 
>> object (not variable, second mistake) must be done in strict 
>> conformity with the abstract machine used to define the semantics of 
>> the C language in the specification. You're not telling the compiler 
>> why you're asking for this (third mistake) and the "value may be 
>> changed in ways not explicitly specified by the program" is only one 
>> of the reasons you might want strict conformance (fourth mistake).
>> 
>>> My friend says "... you need to declare that
>>> variable as volatile, to tell the compiler
>>> to not use the optimized value ..."
>> 
>> What the heck is "the optimized value"? This is nonsense. If you think 
>> you understand it, then that's because you think you already 
>> understand volatile and you're able to map this gobbledygook onto your 
>> understanding.
>> 
>>> So I understand him to be saying Volatile
>>> tells the compiler 1) not to use an optimized
>>> value 2) not to assume that the value hasn't
>>> changed
>> 
>> This is a poor description of volatile for reasons presented above.
>> 
>> I have yet to come across a description of volatile that conforms to 
>> what is written in the standard, is complete, correct and is in all 
>> situations helpful to people who do not fully understand the 
>> terminology of the standard, compilation and software engineering in 
>> general. I am starting to think that nobody would read such a 
>> description as it would necessarily be quite long and people's 
>> preference is for short over complete and correct.
>> 
>> You might think, for example, that Keil (part of ARM) could get this 
>> right. But, they gave the job of writing this description of the 
>> volatile type qualifier:
>> 
>> http://www.keil.com/support/man/docs/c51/c51_le_volatile.htm
>> 
>> to someone who has a pretty good understanding of the C language but 
>> no understanding of hardware. Can you spot the glaring problem?
>> 
>> Graham Davies
>> ECROS Technology
>> www.ecrostech.com
>> 
>> 
> 
> 
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
>

R E Purcella wrote:

> I think you are nit picking words.

Well, yes, of course I'm nit picking words.  I have taken it upon myself to try to get the use of the volatile qualifier straightened out.  The only tool I have to do that is words.  How can I expect to communicate precisely if I don't use words precisely?

For example, take the "unassailable" statement that you should use the volatile qualifier when declaring a variable that represents a special function register.  The only reason that you understand and accept that is that you *misunderstand* the word "variable" or patch up the vague word "represents".  You actually can't declare a variable to directly stand in for an SFR because if you declare a variable the compiler and linker get to choose where to put it and you can't (without using stupid linker tricks) get past that to put the variable where the SFR is.

What you can do, is declare and define a pointer variable, the value of which is initialized to the address of the SRF, and then de-reference that to access the SFR.  Then you do not declare that pointer variable as volatile qualified type (you should rather declare it const).  You declare it as a pointer to a volatile qualified type.  This is important!  If you think that the pointer "represents" the SFR and declare it volatile, instead of the type it points to, you will not get the intended behavior and it will be very hard for you to figure out why.

Alternatively, as in the AVR gcc header files, you can use a pre-processor macro to define a combined type cast and de-reference of a manifest constant and then use that macro as if it were a variable.  But, it isn't a variable.  There is no variable.  We're directly accessing an address in a certain way (according to the type cast) without a variable.

My position, which you can criticize if you like, is that if I'm going to try to explain something I should neither a) assume that the reader already knows enough about that thing to read past any sloppy use of words nor b) expect that the reader will misunderstand a word in the same way that I do.  If I'm going to put an explanation forward, it is my duty to use all reasonable efforts to make sure that explanation is complete, correct and understandable by my target audience.

Having written the above, it now occurs to me that I don't belong on the Internet.

Graham Davies
ECROS Technology
www.ecrostech.com

> If I'm going to put an explanation forward, it is my duty to use all
> reasonable efforts to make sure that explanation is complete, correct and
> understandable by my target audience.
>
> Having written the above, it now occurs to me that I don't belong on the
> Internet.

Obligatory XKCD cartoon:

http://xkcd.com/386/

On 4/20/2012 9:49 PM, Brian Dean wrote:
> The essence of it is that it instructs the compiler that this
> "storage area", is, as the name implies, volatile.Meaning, the
 > content may change outside of the compiler's compile-time knowledge,


Actually it doesn't.  If it did

const volatile int f;

would be invalid, rather than being useful.

Note also that const does not mean constant.

> or, in the case of some hardware ports/registers, accesses to said
> storage have side effects hardware related side-effects (clearing of
> certain bits after a read, and things like that).

That extends the dictionary definition beyond recognition. The 
comparison to the standard english meaning of volatile breaks down here.

Explaining volatile in the context of interrupts and status registers is 
a good introduction but leaving the understanding at that point would be 
a disservice, it leads to the misunderstandings that have been rife in 
this discussion.  Layering additional meaning on top in an "it also 
means a, b, c" fashion makes the definition needlessly complex.

As far as volatile being misnamed it's not the first time programming 
languages have taken a general term and given it a definition that isn't 
strongly related to it's usual definition and it won't be the last.  I 
haven't heard a better suggestion and it's too late in any case.

Actually the definitions of const and volatile leave a gaping hole in 
the qualifications needed for I/O register access.  They provide no way 
to define a write only variable. There no way to inform the compiler 
that a particular variable may not be read.

Robert

-- 

  From the Divided by a Common Language File (Edited to protect the guilty)
ME - "I'd like to get Price and delivery for connector Part # XXXXX"
Dist./Rep - "$X.XX Lead time 37 days"
ME - "Anything we can do about lead time?  37 days seems a bit high."
Dist./Rep - "that is the lead time given because our stock is live....
we currently have stock."

On Apr 21, 2012, at 10:02 AM, Robert Adsett wrote:

> On 4/20/2012 9:49 PM, Brian Dean wrote:
> > The essence of it is that it instructs the compiler that this
> > "storage area", is, as the name implies, volatile.Meaning, the
> > content may change outside of the compiler's compile-time knowledge,
> 
> Actually it doesn't. If it did
> 
> const volatile int f;
> 
> would be invalid, rather than being useful.

Not at all.  Const merely instructs the compiler that the storage that it qualifies is read-only.  It can still be volatile.  The two terms neither conflict nor are inconsistent when used together.  For example, if the hardware implements flags in a register such that a bit is automatically cleared when it is read, and that status register is also read-only, i.e., the programmer cannot write to the status register, then it is both const and volatile.  There is no inconsistency here, and the term volatile certainly applies.  I.e., by it's very nature, just using the dictionary definition of volatile, if simply "looking" at something (reading it) causes it to change, most reasonable people would call that volatile.  To me, that meets the very definition of volatile.  The nature of the status register that it cannot be written to, makes it 'const'.


> Note also that const does not mean constant.

We agree.  But I don't recall saying that it did.


> > or, in the case of some hardware ports/registers, accesses to said
> > storage have side effects hardware related side-effects (clearing of
> > certain bits after a read, and things like that).
> 
> That extends the dictionary definition beyond recognition. The 
> comparison to the standard english meaning of volatile breaks down here.

Not at all.  See explanation above.  As stated above, when dealing with hardware registers as opposed to memory, one does not have actually write to a location for it to change.  We have all dealt with hardware where reading a location's value causes it to change, a fairly common case being status registers where the hardware implementors implicitly clear status bits when the register is read.


> As far as volatile being misnamed it's not the first time programming 
> languages have taken a general term and given it a definition that isn't 
> strongly related to it's usual definition and it won't be the last. I 
> haven't heard a better suggestion and it's too late in any case.

I respectfully disagree.  I have tried to make the case for it above, I think it is a highly accurate keyword that accurately describes the nature of the storage to which it is applied.


> Actually the definitions of const and volatile leave a gaping hole in 
> the qualifications needed for I/O register access. They provide no way 
> to define a write only variable. There no way to inform the compiler 
> that a particular variable may not be read.

This is true, but it is not the fault of 'volatile' or 'const'.  They accurately describe the nature of the storage to which they are applied.  If there is a gap, it is the lack of an additional type qualifier, not the fault of the existing one's because they serve their intended purpose concisely and accurately.  The fact that the C language doesn't have a qualifier for specifying storage that is write-only is unrelated to any deficiency in 'volatile' or 'const'.

-Brian
-- 
Brian Dean
BDMICRO LLC
http://www.bdmicro.com/