Local variable assignments with x appearing both as lval and in the rval fail

[Christian Speckner]

This one is a bit complicated to explain :) If I have a variable assignment in a local scope with x appearing both as lval and in the rval, the evaluation in the rval fails. An example:

int test = 1
test = test + 1
printf "test: %i" (test)

printf "Hello World" {
   test = test + 1
   printf "test: %i" (test)
}

scan int dummy = (1) {
   test = test + 1
   printf "test: %i" (test)
}

scan int dummy = (1) {
   int temp = test
   test = temp + 1
   printf "test: %i" (test)
}

The output of this is

[user variable] test =            1
[user variable] test =            2
test: 2
[user variable] test = [unknown integer]

Hello World
[user variable] dummy =            1
[user variable] test = [unknown integer]

[user variable] dummy =            1
[user variable] temp =            2
[user variable] test =            3
test: 3

The first increment in the local scope succeeds, but the next two increments in the local scopes of printf and scan fail. The third one, although also in a local scope, works because of the detour via a temporary variable.

[Christian Speckner]

Furthermore, cmd_var does not seem to play along with "compile once, execute more the once" (aka loops): the script

scan int dummy = (1, 2, 3) {
   int temp = test
   test = temp + 1
   printf "test: %i" (test)
}

returns

[user variable] test =            1
[user variable] dummy =            1
[user variable] temp =            1
[user variable] test =            2
test: 2
[user variable] dummy =            2
[user variable] temp =            1
[user variable] test =            2
test: 2
[user variable] dummy =            3
[user variable] temp =            1
[user variable] test =            2
test: 2

I'm pretty sure this issue is not rooted in cmd_scan because I am getting the same behavior when doing such reassignments in my plotting macros.

[Christian Speckner]

The following works as expected:

scan int dummy = (1, 2, 3) {
   if (dummy == 1) then
      int test = 1
   else
      int test = test + 1
   endif
   printf "test: %i" (test)
}

[kilian]

I also just checked this and confirm it.

I could fix the original problem: the rhs is evaluated before the lhs (correct), but before that, the rhs is compiled _after_ the lhs, so it tries to refer to the just created local variable instead of the global one.

[kilian]

However, the scan shows a design flaw of the implementation: in the first loop eval, it should take the global 'test' as rhs, but in subsequent evals, it should take the local 'test'. This is simply not possible in the present setup. I have to think about this.

You also see a problem if, inside the scan, you put a 'show (test)' before the first reference to test. It will show the local, not the global var.

[Christian Speckner]

OK, I think I've figures it out. There are two issues here, both of which aren't actually bugs in the sense of the word but still (at least in my opinion) highly counterintuitive. The first issue arises because the variable declaration first creates a local instance of test and the compiles the rval, with the compiled expression now containing a pointer to the local, uninitialized variable. The second one comes from the fact that int dummy = test is compiled before int test = dummy + 1 and therefore contains a pointer to the global variable.

I think the first point can (and should) be fixed by reversing the order of compilation in variable assiglments (first rval, then lval); the second one makes sense as-is but needs to be documented.

P.S.: Just noticed that Wolfgang beat me at the conclusion while I was typing ;)

[Christian Speckner]

What also comes to mind: it would be nice to allow for declaring variables without actually assigning a value; otherwise, some constructions in local scopes require elaborate tricks or are simply impossible --- this severely affects the flexibility of my write_analysis_data implementation (c.f. #48).

[Christian Speckner]

In r2543, I've taken the liberty of reversing the order of compilation in variable assignments - it's now first rval, then lval. This solves the original issue. Furthermore, I'm looking into implementing a "declaration" mode for variable assignments by allowing for statements like

declare int i = 1

This kind of statement would be executed only once and skipped on subsequent execution (e.g. in a loop or a plot / histogram writer).

[Christian Speckner]

In r2546, I've implemented a new statement

once command | { command_list }

which executes the command(s) within the current scope once; repeated executions of the same command(s) are ignored. This can be used in loops, histogram / plot writers (and possibly in future user-defined macros and functions) to define a set of initialized local variables. As an example, the script

int var = 5
scan int i_ = (1 => 6 + 1) {
   once {
      printf "Starting loop..."
      printf "Initializing var with %i" (var)
      once var = var
   }
   var = var + var
}
print (var)

returns

[user variable] var =            5                                                                                                         
[user variable] i_ =            1                                                                                                          
Starting loop...                                                                                                                           
Initializing var with 5
[user variable] var =            5
[user variable] var =           10
[user variable] i_ =            2
[user variable] var =           20
[user variable] i_ =            3
[user variable] var =           40
[user variable] i_ =            4
[user variable] var =           80
[user variable] i_ =            5
[user variable] var =          160
[user variable] i_ =            6
[user variable] var =          320
[user variable] i_ =            7
[user variable] var =          640
5

With this addition, my original goal of using local variables to access the previous plot points / histogram bins in the custom plot / histogram writers can be attained easily, so, if WK agrees, I'm happy with closing this ticket ;)

[Christian Speckner]

Whoops, just got the mail with my own edit and spotted a typo: The second once is spurious :) Should be

int var = 5
scan int i_ = (1 => 6 /+ 1) {
   once {
      printf "Starting loop..."
      printf "Initializing var with %i" (var)
      var = var
   }
   var = var + var
}
print (var)

(would work nevertheless).

[kilian]

Replying to cnspeckn:

In r2546, I've implemented a new statement

once command | { command_list }

which executes the command(s) within the current scope once; repeated executions of the same command(s) are ignored. This can be used in loops, histogram / plot writers (and possibly in future user-defined macros and functions) to define a set of initialized local variables. As an example, the script

That's a solution, but I'd rather restrict the language to as few keywords as possible, in particular since such a construct is not a familiar one; it rather forces the user to solve a problem of the implementation. And it only partially solves the problem; there are other related issues with the implementation of local variables.

There must be a way such that the loops and variables rather do what the user expects. I've got some ideas (that's why I didn't commit the reordering of rhs and lhs compilation, which I also had as a quick fix), but it's not done yet.

For the moment, keep 'once' as an undocumented feature which eventually should become obsolete.

[Christian Speckner]

I'm fine with any other working, documented solution.

[kilian]

I have a solution that almost works .. need some testing and fine-tuning.

[kilian]

Oops, sorry, r2565 broke something else (make check in trunk broken!) -- will fix it asap.

[kilian]

The handling of variables is now conforming to their use in local command lists. After the local list is compiled, the variables get an undefined state. During execution, if the variable is referenced before definition, the program skips the local entry and tries to fetch the value from global scope instead. Only after a variable is defined, the value is taken from local scope. This requires recompiling the rhs of a variable definition each time it is executed again, which happens in loops. (Local parameters defined by the 'let' construct are not affected, they never can be defined in terms of themselves.) When leaving local scope (i.e., after all iterations in a loop have been completed), variables become undefined again.

Done in r2566.

Here is the previous example, now without the 'once':

int var = 5
scan int i_ = (1 => 3 /+ 1) {
   var = var + var
}
print (var)

with the output

[user variable] var =  5
[user variable] i_ =  1
[user variable] var =  10
[user variable] i_ =  2
[user variable] var =  20
[user variable] i_ =  3
[user variable] var =  40
5

Here is another example which is a bit tricky:

int i = 1
scan int loop_1 = (1, 2) {
  scan int loop_2 = (1, 2) {
    i = i + 3
  }
  scan int loop_3 = (1+i, 2+i)
  i = i + 7
}

The output:

[user variable] i =  1
[user variable] loop_1 =  1
[user variable] loop_2 =  1
[user variable] i =  4
[user variable] loop_2 =  2
[user variable] i =  7
[user variable] loop_3 =  2
[user variable] loop_3 =  3
[user variable] i =  8
[user variable] loop_1 =  2
[user variable] loop_2 =  1
[user variable] i =  11
[user variable] loop_2 =  2
[user variable] i =  14
[user variable] loop_3 =  9
[user variable] loop_3 =  10
[user variable] i =  15