PDL::PP - Man Page

Add code to the BOOT section of generated XS file

Add functions to the list of exported functions

Add entries to the @ISA list

Sets code to be added at the top of the generate .pm file

Add code and includes to C section of the generated XS file.

When used in a module that is "multi-C" (one .c file per pp_defed function), you need to bear in mind that as each one is generated, all the pp_addhdr so far will be included. Therefore, if you add C functions, make sure to make them static to avoid clashes with later .c files, or add the C functions to the CHeader key (available as of version 2.086) of "pp_def".

Another alternative is to make them be separate C files, with any necessary .h to be included by them and the .pd file. You can then add them to your Makefile.PL (note this is the _int version, see separate notes on how to "opt-in" for your own modules):

  my @pack = (["pnm.pd", qw(Pnm PDL::IO::Pnm)]);
  my %hash = pdlpp_stdargs_int(@pack);
  $hash{OBJECT} .= ' get$(OBJ_EXT)';
  sub MY::postamble { pdlpp_postamble_int(@pack); }
  WriteMakefile(%hash);

Add code to the generated .pm file

Add extra XS code to the generated XS file

Add extra $MACRO() definitions for these functions. Note these generate C code. As of 2.080, they will be passed the list of arguments they were called with, rather than a single string, split like the C pre-processor on commas except if in "" or (), with leading and trailing whitespace removed.

  pp_add_macros(SUCC => sub { "($_[0] + 1)" });
  # ...
    Code => '$a() = $SUCC($b());',

Available from 2.082. Add an XS typemap for use as OtherPars or from manually-added XS. Takes one named argument, either typemap (an ExtUtils::Typemaps object), string, or file.

  pp_add_typemaps(string=><<'EOT');
  TYPEMAP
  NV_ADD1 T_NV_ADD1

  INPUT
  T_NV_ADD1
    $var = SvNV($arg) + 1;

  OUTPUT
  T_NV_ADD1
    sv_setnv($arg, $var - 1);
  EOT
  # ...
    OtherPars => '[o] NV_ADD1 v1',

Add BEGIN-block wrapping to code for the generated .pm file

Sets the package to which the XS code is added (default is PDL)

Specify what is imported from PDL::Core

Define a new PDL function

Add runtime and POD warnings about a module being deprecated

Mark the end of PDL::PP definitions in the file

Clear out the export list for your generated module

Add line number information to simplify debugging of PDL::PP code

In the data operation, you must know what dimensions of data you need. First, an example with scalars:

        pp_def('add',
                Pars => 'a(); b(); [o]c();',
                Code => '$c() = $a() + $b();'
        );

That looks a little strange but let's dissect it. The first line is easy: we're defining a routine with the name 'add'. The second line simply declares our parameters and the parentheses mean that they are scalars. We call the string that defines our parameters and their dimensionality the signature of that function. For its relevance with regard to broadcasting and index manipulations check the PDL::Indexing man page.

The third line is the actual operation. You need to use the dollar signs and parentheses to refer to your parameters (this will probably change at some point in the future, once a good syntax is found).

These lines are all that is necessary to actually define the function for PDL (well, actually it isn't; you additionally need to write a Makefile.PL (see below) and build the module (something like 'perl Makefile.PL; make'); but let's ignore that for the moment). So now you can do

        use MyModule;
        $x = pdl 2,3,4;
        $y = pdl 5;

        $c = add($x,$y);
        # or
        add($x,$y,($c=null)); # Alternative form, useful if $c has been
                              # preset to something big, not useful here.

and have broadcasting work correctly (the result is $c == [7 8 9]).

Seeing the above example code you will most probably ask: what is this strange $c=null syntax in the second call to our new add function? If you take another look at the definition of add you will notice that the third argument c is flagged with the qualifier [o] which tells PDL::PP that this is an output argument. So the above call to add means 'create a new $c from scratch with correct dimensions' - null is a special token for 'empty ndarray' (you might ask why we haven't used the value undef to flag this instead of the PDL specific null; we are currently thinking about it ;).

[This should be explained in some other section of the manual as well!!] The reason for having this syntax as an alternative is that if you have really huge ndarrays, you can do

        $c = PDL->null;
        for(some long loop) {
                # munge a,b
                add($x,$y,$c);
                # munge c, put something back to x,y
        }

and avoid allocating and deallocating $c each time. It is allocated once at the first add() and thereafter the memory stays until $c is destroyed.

If you just say

  $c =  add($x,$y);

the code generated by PP will automatically fill in $c=null and return the result. If you want to learn more about the reasons why PDL::PP supports this style where output arguments are given as last arguments check the PDL::Indexing man page.

[o] is not the only qualifier a pdl argument can have in the signature. Another important qualifier is the [t] option which flags a pdl as temporary.  What does that mean? You tell PDL::PP that this pdl is only used for temporary results in the course of the calculation and you are not interested in its value after the computation has been completed. But why should PDL::PP want to know about this in the first place?  The reason is closely related to the concepts of pdl auto creation (you heard about that above) and implicit broadcasting. If you use implicit broadcasting the dimensionality of automatically created pdls is actually larger than that specified in the signature. With [o] flagged pdls will be created so that they have the additional dimensions as required by the number of implicit broadcast dimensions. When creating a temporary pdl, however, it will always only be made big enough so that it can hold the result for one iteration in a broadcast loop, i.e. as large as required by the signature. So less memory is wasted when you flag a pdl as temporary. Secondly, you can use output auto creation with temporary pdls even when you are using explicit broadcasting which is forbidden for normal output pdls flagged with [o] (see PDL::Indexing).

As of 2.073, the user is unable to pass a [t] parameter, and PDL will create and size it to its notional size, times the number of threads.

Here is an example where we use the [t] qualifier. We define the function callf that calls a C routine f which needs a temporary array of the same size and type as the array a (sorry about the forward reference for $P; it's a pointer access, see below) :

  pp_def('callf',
        Pars => 'a(n); [t] tmp(n); [o] b()',
        Code => 'PDL_Indx ns = $SIZE(n);
                 f($P(a),$P(b),$P(tmp),ns);
                '
  );

Another possible qualifier is [phys]. If given, this means the pdl will have "make_physical" in PDL::Core called on it.

Additionally, if it has a specified dimension d that has value 1, d will not magically be grown if d is larger in another pdl with specified dimension d, and instead an exception will be thrown. E.g.:

  pp_def('callf',
        Pars => 'a(n); [phys] b(n); [o] c()',
        # ...
  );

If a had lead dimension of 2 and b of 3, an exception will always be thrown. However, if b has lead dimension of 1, it would be silently repeated as if it were 2, if it were not a phys parameter.

Now we have just talked about dimensions of pdls and the signature. How are they related? Let's say that we want to add a scalar + the index number to a vector:

        pp_def('add2',
                Pars => 'a(n); b(); [o]c(n);',
                Code => 'loop(n) %{
                                $c() = $a() + $b() + n;
                         %}'
        );

There are several points to notice here: first, the Pars argument now contains the n arguments to show that we have a single dimensions in a and c. It is important to note that dimensions are actual entities that are accessed by name so this declares a and c to have the same first dimensions. In most PP definitions the size of named dimensions will be set from the respective dimensions of non-output pdls (those with no [o] flag) but sometimes you might want to set the size of a named dimension explicitly through an integer parameter. See below in the description of the OtherPars section how that works.

Suppose you want an output ndarray to be created automatically and you know that on every call its dimension will have the same size (say 9) regardless of the dimensions of the input ndarrays. In this case you use the following syntax in the Pars section to specify the size of the dimension:

    ' [o] y(n=9); '

As expected, extra dimensions required by broadcasting will be created if necessary. If you need to assign a named dimension according to a more complicated formula (than a constant) you can use the RedoDimsCode key described below, or the new (as of 2.088) mechanism of dimname=CALC(1+2+3) etc.

The signature also determines the type conversions that will be performed when a PP function is invoked. So what happens when we invoke one of our previously defined functions with pdls of different type, e.g.

  add2($x,$y,($ret=null));

where $x is of type PDL_Float and $y of type PDL_Short? With the signature as shown in the definition of add2 above the datatype of the operation (as determined at runtime) is that of the pdl with the 'highest' type (sequence is byte < short < ushort < long < float < double). In the add2 example the datatype of the operation is float ($x has that datatype). All pdl arguments are then type converted to that datatype (they are not converted inplace but a copy with the right type is created if a pdl argument doesn't have the type of the operation). Null pdls don't contribute a type in the determination of the type of the operation. However, they will be created with the datatype of the operation; here, for example, $ret will be of type float. You should be aware of these rules when calling PP functions with pdls of different types to take the additional storage and runtime requirements into account.

These type conversions are correct for most functions you normally define with pp_def. However, there are certain cases where slightly modified type conversion behaviour is desired. For these cases additional qualifiers in the signature can be used to specify the desired properties with regard to type conversion. These qualifiers can be combined with those we have encountered already (the creation qualifiers [o] and [t]). Let's go through the list of qualifiers that change type conversion behaviour.

The most important is the indx qualifier which comes in handy when a pdl argument represents indices into another pdl. Let's take a look at an example from PDL::Ufunc:

   pp_def('maximum_ind',
          Pars => 'a(n); indx [o] b()',
          Code => '$GENERIC() cur;
                   PDL_Indx curind;
                   loop(n) %{
                    if (!n || $a() > cur) {cur = $a(); curind = n;}
                   %}
                   $b() = curind;',
   );

The function maximum_ind finds the index of the largest element of a vector. If you look at the signature you notice that the output argument b has been declared with the additional indx qualifier. This has the following consequences for type conversions: regardless of the type of the input pdl a the output pdl b will be of type PDL_Indx which makes sense since b will represent an index into a.

Note that 'curind' is declared as type PDL_Indx and not indx. While most datatype declarations in the 'Pars' section use the same name as the underlying C type, indx is a type which is sufficient to handle PDL indexing operations.  For 32-bit installs, it can be a 32-bit integer type.  For 64-bit installs, it will be a 64-bit integer type.

Furthermore, if you call the function with an existing output pdl b its type will not influence the datatype of the operation (see above). Hence, even if a is of a smaller type than b it will not be converted to match the type of b but stays untouched, which saves memory and CPU cycles and is the right thing to do when b represents indices. Also note that you can use the 'indx' qualifier together with other qualifiers (the [o] and [t] qualifiers). Order is significant -- type qualifiers precede creation qualifiers ([o] and [t]).

The above example also demonstrates typical usage of the $GENERIC() macro.  It expands to the current type in a so called generic loop. What is a generic loop? As you already heard a PP function has a runtime datatype as determined by the type of the pdl arguments it has been invoked with.  The PP generated XS code for this function therefore contains a switch like switch (type) {case PDL_Byte: ... case PDL_Double: ...} that selects a case based on the runtime datatype of the function (it's called a type “loop” because there is a loop in PP code that generates the cases). In any case your code is inserted once for each PDL type into this switch statement. The $GENERIC() macro just expands to the respective type in each copy of your parsed code in this switch statement, e.g., in the case PDL_Byte section cur will expand to PDL_Byte and so on for the other case statements. I guess you realise that this is a useful macro to hold values of pdls in some code.

There are a couple of other qualifiers with similar effects as indx. For your convenience there are the float and double qualifiers with analogous consequences on type conversions as indx. Let's assume you have a very large array for which you want to compute row and column sums with an equivalent of the sumover function. However, with the normal definition of sumover you might run into problems when your data is, e.g. of type short. A call like

  sumover($large_pdl,($sums = null));

will result in $sums be of type short and is therefore prone to overflow errors if $large_pdl is a very large array. On the other hand calling

  @dims = $large_pdl->dims; shift @dims;
  sumover($large_pdl,($sums = zeroes(double,@dims)));

is not a good alternative either. Now we don't have overflow problems with $sums but at the expense of a type conversion of $large_pdl to double, something bad if this is really a large pdl. That's where double comes in handy:

  pp_def('sumoverd',
         Pars => 'a(n); double [o] b()',
         Code => 'double tmp=0;
                  loop(n) %{ tmp += a(); %}
                  $b() = tmp;',
  );

This gets us around the type conversion and overflow problems. Again, analogous to the indx qualifier double results in b always being of type double regardless of the type of a without leading to a type conversion of a as a side effect.

There is also a special type, real. The others above are all actual PDL/C datatypes, but real is a modifier; if the operation type is real, it has no effect; if it is complex, then the parameter will be the real version - so cdouble becomes double, etc.

There is also the converse, complex. If the operation is already complex, there is no effect; if not, the output will be promoted to the type's "complexversion" in PDL::Type, which defaults to cfloat. Note this is controlled both by the PDL::Types data, and the code in PDL::PP. NB Because this outputs floating-point data, the inputs will by definition be turned into such. Therefore, it only makes sense to have floating-point GenericTypes inputs. If you want to default to coercing inputs to float, give that as the last GenericTypes as the generated XS function defaults to the last-given one. Hence (with the PMCode and Doc omitted):

  pp_def('r2C',
    GenericTypes=>[reverse qw(F D G C)], # last one is default so here = F
    Pars => 'r(); complex [o]c()',
    Code => '$c() = $r();'
  );

Finally, there are the type+ qualifiers where type is one of int or float. What shall that mean. Let's illustrate the int+ qualifier with the actual definition of sumover:

  pp_def('sumover',
         Pars => 'a(n); int+ [o] b()',
         Code => '$GENERIC(b) tmp=0;
                  loop(n) %{ tmp += a(); %}
                  $b() = tmp;',
  );

As we had already seen for the int, float and double qualifiers, a pdl marked with a type+ qualifier does not influence the datatype of the pdl operation. Its meaning is "make this pdl at least of type type or higher, as required by the type of the operation". In the sumover example this means that when you call the function with an a of type PDL_Short the output pdl will be of type PDL_Long (just as would have been the case with the int qualifier). This again tries to avoid overflow problems when using small datatypes (e.g. byte images).  However, when the datatype of the operation is higher than the type specified in the type+ qualifier b will be created with the datatype of the operation, e.g. when a is of type double then b will be double as well. We hope you agree that this is sensible behaviour for sumover. It should be obvious how the float+ qualifier works by analogy. It may become necessary to be able to specify a set of alternative types for the parameters. However, this will probably not be implemented until someone comes up with a reasonable use for it.

Note that we now had to specify the $GENERIC macro with the name of the pdl to derive the type from that argument. Why is that? If you carefully followed our explanations you will have realised that in some cases b will have a different type than the type of the operation. Calling the '$GENERIC' macro with b as argument makes sure that the type will always the same as that of b in that part of the generic loop.

This is about all there is to say about the Pars section in a pp_def call. You should remember that this section defines the signature of a PP defined function, you can use several options to qualify certain arguments as output and temporary args and all dimensions that you can later refer to in the Code section are defined by name.

It is important that you understand the meaning of the signature since in the latest PDL versions you can use it to define broadcasting functions from within Perl, i.e. what we call Perl level broadcasting. Please check PDL::Indexing for details.

The Code section contains the actual XS code that will be in the innermost part of a broadcast loop (if you don't know what a broadcast loop is then you still haven't read PDL::Indexing; do it now ;) after any PP macros (like $GENERIC) and PP functions have been expanded (like the loop function we are going to explain next).

Let's quickly reiterate the sumover example:

  pp_def('sumover',
         Pars => 'a(n); int+ [o] b()',
         Code => '$GENERIC(b) tmp=0;
                  loop(n) %{ tmp += a(); %}
                  $b() = tmp;',
  );

The loop construct in the Code section also refers to the dimension name so you don't need to specify any limits: the loop is correctly sized and everything is done for you, again.

Next, there is the surprising fact that $a() and $b() do not contain the index. This is not necessary because we're looping over n and both variables know which dimensions they have so they automatically know they're being looped over.

This feature comes in very handy in many places and makes for much shorter code. Of course, there are times when you want to circumvent this; here is a function which make a matrix symmetric and serves as an example of how to code explicit looping:

        pp_def('symm',
                Pars => 'a(n,n); [o]c(n,n);',
                Code => 'loop(n) %{
                                int n2;
                                for(n2=n; n2<$SIZE(n); n2++) {
                                        $c(n0 => n, n1 => n2) =
                                        $c(n0 => n2, n1 => n) =
                                         $a(n0 => n, n1 => n2);
                                }
                        %}
                '
        );

Let's dissect what is happening. Firstly, what is this function supposed to do? From its signature you see that it takes a 2D matrix with equal numbers of columns and rows and outputs a matrix of the same size. From a given input matrix $a it computes a symmetric output matrix $c (symmetric in the matrix sense that A^T = A where ^T means matrix transpose, or in PDL parlance $c == $c->transpose). It does this by using only the values on and below the diagonal of $a. In the output matrix $c all values on and below the diagonal are the same as those in $a while those above the diagonal are a mirror image of those below the diagonal (above and below are here interpreted in the way that PDL prints 2D pdls). If this explanation still sounds a bit strange just go ahead, make a little file into which you write this definition, build the new PDL extension (see section on Makefiles for PP code) and try it out with a couple of examples.

Having explained what the function is supposed to do there are a couple of points worth noting from the syntactical point of view. First, we get the size of the dimension named n again by using the $SIZE macro. Second, there are suddenly these funny n0 and n1 index names in the code though the signature defines only the dimension n. Why this? The reason becomes clear when you note that both the first and second dimension of $a and $b are named n in the signature of symm. This tells PDL::PP that the first and second dimension of these arguments should have the same size. Otherwise the generated function will raise a runtime error. However, now in an access to $a and $c PDL::PP cannot figure out which index n refers to any more just from the name of the index. Therefore, the indices with equal dimension names get numbered from left to right starting at 0, e.g. in the above example n0 refers to the first dimension of $a and $c, n1 to the second and so on.

In all examples so far, we have only used the Pars and Code members of the hash that was passed to pp_def. There are certainly other keys that are recognised by PDL::PP and we will hear about some of them in the course of this document. Find a (non-exhaustive) list of keys in Appendix A.  A list of macros and PPfunctions (we have only encountered some of those in the examples above yet) that are expanded in values of the hash argument to pp_def is summarised in Appendix B.

At this point, it might be appropriate to mention that PDL::PP is not a completely static, well designed set of routines (as Tuomas puts it: "stop thinking of PP as a set of routines carved in stone") but rather a collection of things that the PDL::PP author (Tuomas J. Lukka) considered he would have to write often into his PDL extension routines. PP tries to be expandable so that in the future, as new needs arise, new common code can be abstracted back into it. If you want to learn more on why you might want to change PDL::PP and how to do it check the section on PDL::PP internals.

There are several keys and macros used when writing code to handle bad values. The first one is the HandleBad key:

HandleBad => 0

This flags a pp-routine as NOT handling bad values. If this routine is sent ndarrays with their badflag set, then a warning message is printed to STDOUT and the ndarrays are processed as if the value used to represent bad values is a valid number. The badflag value is not propagated to the output ndarrays.

An example of when this is used is for FFT routines, which generally do not have a way of ignoring part of the data.

HandleBad => 1

This causes PDL::PP to write extra code that ensures the BadCode section is used, and that the $ISBAD() macro (and its brethren) work. If no BadCode is supplied, the Code section will be used, on the assumption it will use PDL_IF_BAD to handle bad values.

HandleBad is not given

If any of the input ndarrays have their badflag set, then the output ndarrays will have their badflag set, but any supplied BadCode is ignored.

The value of HandleBad is used to define the contents of the BadDoc key, if it is not given.

To handle bad values, code must be written somewhat differently; for instance,

 $c() = $a() + $b();

becomes something like

 if ( $a() != BADVAL && $b() != BADVAL ) {
    $c() = $a() + $b();
 } else {
    $c() = BADVAL;
 }

However, we only want the second version if bad values are present in the input ndarrays (and that bad-value support is wanted!) - otherwise  we actually want the original code. This is where the BadCode key comes in; you use it to specify the code to execute if bad values may be present, and PP uses both it and the Code section to create something like:

 if ( bad_values_are_present ) {
    fancy_broadcastloop_stuff {
       BadCode
    }
 } else {
    fancy_broadcastloop_stuff {
       Code
    }
 }

This approach means that there is virtually no overhead when  bad values are not present (i.e. the badflag routine returns 0).

The C preprocessor symbol PDL_BAD_CODE is defined when the bad code is compiled, so that you can reduce the amount of code you write.  The BadCode section can use the same macros and looping constructs as the Code section. As of 2.073, you can also use PDL_IF_BAD(iftrue,iffalse).

However, it wouldn't be much use without the following additional macros:

$ISBAD(var)

To check whether an ndarray's value is bad, use the $ISBAD macro:

 if ( $ISBAD(a()) ) { printf("a() is bad\n"); }

You can also access given elements of an ndarray:

 if ( $ISBAD(a(n=>l)) ) { printf("element %d of a() is bad\n", l); }

$ISGOOD(var)

This is the opposite of the $ISBAD macro.

$SETBAD(var)

For when you want to set an element of an ndarray bad.

$ISBADVAR(c_var,pdl)

If you have cached the value of an ndarray $a() into a c-variable (foo say), then to check whether it is bad, use $ISBADVAR(foo,a).

$ISGOODVAR(c_var,pdl)

As above, but this time checking that the cached value isn't bad.

$SETBADVAR(c_var,pdl)

To copy the bad value for an ndarray into a c variable, use $SETBADVAR(foo,a).

TODO: mention $PISBAD() etc macros.

If you want access to the value of the badflag for a given ndarray, you can use the PDL STATE macros, for use in "CopyBadStatusCode".

$ISPDLSTATEBAD(pdl)

$ISPDLSTATEGOOD(pdl)

$SETPDLSTATEBAD(pdl)

$SETPDLSTATEGOOD(pdl)

And for use in Code sections:

$PDLSTATEISBAD(pdl)

$PDLSTATEISGOOD(pdl)

$PDLSTATESETBAD(pdl)

$PDLSTATESETGOOD(pdl)

Using these macros, the above code could be specified as:

 Code => '$c() = $a() + $b();',
 BadCode => '
    if ( $ISBAD(a()) || $ISBAD(b()) ) {
       $SETBAD(c());
    } else {
       $c() = $a() + $b();
    }',

Since this is Perl, TMTOWTDI, so you could also write:

 BadCode => '
    if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
       $c() = $a() + $b();
    } else {
       $SETBAD(c());
    }',

You can reduce code repetition using the C PDL_BAD_CODE macro, supplying only the Code section:

 Code => '
    #ifdef PDL_BAD_CODE
    if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
    #endif PDL_BAD_CODE
       $c() = $a() + $b();
    #ifdef PDL_BAD_CODE
    } else {
       $SETBAD(c());
    }
    #endif PDL_BAD_CODE
 ',

As of 2.073, you can also use PDL_IF_BAD(iftrue,iffalse):

 Code => '
    PDL_IF_BAD(if ( $ISGOOD(a()) && $ISGOOD(b()) ) {,)
       $c() = $a() + $b();
    PDL_IF_BAD(} else $SETBAD(c());,)
 ',

Now, consider the following: you have your own C function (that may in fact be part of some library you want to interface to PDL) which takes as arguments two pointers to vectors of double:

        void myfunc(int n,double *v1,double *v2);

The correct way of defining the PDL function is

        pp_def('myfunc',
                Pars => 'a(n); [o]b(n);',
                GenericTypes => ['D'],
                Code => 'myfunc($SIZE(n),$P(a),$P(b));'
        );

The $P(par) syntax returns a pointer to the first element and the other elements are guaranteed to lie after that. Using it in your Code guarantees it will be "physicalised", so that all the data is contiguous, because external C functions usually expect that.

Notice that here it is possible to make many mistakes. First, $SIZE(n) must be used instead of n. Second, you shouldn't put any loops in this code. Third, here we encounter a new hash key recognised by PDL::PP : the GenericTypes declaration tells PDL::PP to ONLY GENERATE THE TYPELOOP FOP THE LIST OF TYPES SPECIFIED. In this case double. This has two advantages. Firstly the size of the compiled code is reduced vastly, secondly if non-double arguments are passed to myfunc() PDL will automatically convert them to double before passing to the external C routine and convert them back afterwards.

One can also use Pars to qualify the types of individual arguments. Thus one could also write this as:

        pp_def('myfunc',
                Pars => 'double a(n); double [o]b(n);',
                Code => 'myfunc($SIZE(n),$P(a),$P(b));'
        );

The type specification in Pars exempts the argument from variation in the typeloop - rather it is automatically converted to and from the type specified. This is obviously useful in a more general example, e.g.:

        void myfunc(int n,float *v1,long *v2);

        pp_def('myfunc',
                Pars => 'float a(n); long [o]b(n);',
                GenericTypes => ['F'],
                Code => 'myfunc($SIZE(n),$P(a),$P(b));'
        );

Note we still use GenericTypes to reduce the size of the type loop, obviously PP could in principle spot this and do it automatically though the code has yet to attain that level of sophistication!

Finally note when types are converted automatically one MUST use the [o] qualifier for output variables or you hard-won changes will get optimised away by PP!

If you interface a large library you can automate the interfacing even further. Perl can help you again(!) in doing this. In many libraries you have certain calling conventions. This can be exploited. In short, you can write a little parser (which is really not difficult in Perl) that then generates the calls to pp_def from parsed descriptions of the functions in that library. For an example, please check the Slatec interface in the Lib tree of the PDL distribution. If you want to check (during debugging) which calls to PP functions your Perl code generated a little helper package comes in handy which replaces the PP functions by identically named ones that dump their arguments to stdout.

Just say

   perl -MPDL::PP::Dump myfile.pd

to see the calls to pp_def and friends. Try it with ops.pd and slatec.pd. If you're interested (or want to enhance it), the source is in Basic/Gen/PP/Dump.pm

Macros: So far we have encountered the $SIZE, $GENERIC and $P macros. Now we are going to quickly explain the other macros that are expanded in the Code section of PDL::PP along with examples of their usage.

$T

The $T macro is used for type switches. This is very useful when you have to use different external (e.g. library) functions depending on the input type of arguments. The general syntax is

        $Ttypeletters(type_alternatives)

where typeletters is a permutation of a subset of the letters BSULNQFDGC which stand for Byte, Short, Ushort, etc. and type_alternatives are the expansions when the type of the PP operation is equal to that indicated by the respective letter. Let's illustrate this incomprehensible description by an example. Assuming you have two C functions with prototypes

  void float_func(float *in, float *out);
  void double_func(double *in, double *out);

which do basically the same thing but one accepts float and the other double pointers. You could interface them to PDL by defining a generic function foofunc (which will call the correct function depending on the type of the transformation):

  pp_def('foofunc',
        Pars => ' a(n); [o] b();',
        Code => ' $TFD(float,double)_func ($P(a),$P(b));'
        GenericTypes => [qw(F D)],
  );

There is a limitation that the comma-separated values cannot have parentheses.

$PPSYM

The $PPSYM() macro is replaced by the value of "ppsym" in PDL::Types for the loop type, or that of the given parameter, similar to $GENERIC(). This is useful for e.g. macros that vary by that, avoiding the need for things like $TXY(X,Y). Another benefit is that if an operation's GenericTypes get extended, this macro will still be correct.

$COMP (and the OtherPars section)

The $COMP macro is used to access non-pdl values in the code section. Its name is derived from the implementation of transformations in PDL. The variables you can refer to using $COMP are members of the “compiled” structure that represents the PDL transformation in question but does not yet contain any information about dimensions (for further details check PDL::Internals). However, you can treat $COMP just as a black box without knowing anything about the implementation of transformations in PDL. So when would you use this macro? Its main usage is to access values of arguments that are declared in the OtherPars section of a pp_def definition. But then you haven't heard about the OtherPars key yet?!  Let's have another example that illustrates typical usage of both new features:

  pp_def('pnmout',
        Pars => 'a(m)',
        OtherPars => "PerlIO *fp",
        GenericTypes => [qw(B U S L)],
        Code => '
                 if (PerlIO_write($COMP(fp),$P(a),len) != len)
                                $CROAK("Error writing pnm file");
  ');

This function is used to write data from a pdl to a file. The file descriptor is passed as a string into this function. This parameter does not go into the Pars section since it cannot be usefully treated like a pdl but rather into the aptly named OtherPars section. Parameters in the OtherPars section follow those in the Pars section when invoking the function, i.e.

   open FILE,">out.dat" or die "couldn't open out.dat";
   pnmout($pdl,'FILE');

When you want to access this parameter inside the code section you have to tell PP by using the $COMP macro, i.e. you write $COMP(fp) as in the example. Otherwise PP wouldn't know that the fp you are referring to is the same as that specified in the OtherPars section.

Another use for the OtherPars section is to set a named dimension in the signature. Let's have an example how that is done:

  pp_def('setdim',
        Pars => '[o] a(n)',
        OtherPars => 'int ns => n',
        Code => 'loop(n) %{ $a() = n; %}',
  );

This says that the named dimension n will be initialised from the value of the other parameter ns which is of integer type (I guess you have realised that we use the CType From => named_dim syntax). As of 2.082, this can be used to set the size of a dimension not used in any Pars. Now you can call this function in the usual way:

  setdim(($x=null),5);
  print $x;
    [ 0 1 2 3 4 ]

Admittedly this function is not very useful but it demonstrates how it works. If you call the function with an existing pdl and you don't need to explicitly specify the size of n since PDL::PP can figure it out from the dimensions of the non-null pdl. In that case you just give the dimension parameter as -1:

  $x = hist($y);
  setdim($x,-1);

The default values available via $COMP() are the OtherPars as noted above, which get copied in. However, this can be added to (previous to 2.058, replaced) by supplying Comp and/or MakeComp keys (the defaults will happen first):

  pp_def(
    'diagonal',
    OtherPars => 'SV *list',
    Comp => 'PDL_Indx whichdims_count; PDL_Indx whichdims[$COMP(whichdims_count)];',
    MakeComp => '
      PDL_Indx i;
      PDL_Indx *tmp= PDL->packdims(list,&($COMP(whichdims_count)));
      if (!tmp) $CROAK("Failed to packdims for creating");
      if ($COMP(whichdims_count) < 1)
        $CROAK("Diagonal: must have at least 1 dimension");
      $DOCOMPALLOC(); /* malloc()s the whichdims */
      for(i=0; i<$COMP(whichdims_count); i++)
        $COMP(whichdims)[i] = tmp[i];
      free(tmp);
      /* ... */
    ',
    # ...
  );

The MakeComp code is placed in the pdl_run_(funcname), so access to Pars (which will just be pdl *s)/OtherPars values is just via their names, not a macro. The default code (which also applies to OtherPars) makes a copy of values where it knows how to do so, including SV* and char*.

You can also provide a CompFreeCodeComp key, in case your MakeComp needs tidying up after it.

As of 2.058, you can instead give a C99 "incomplete array" type parameter as an OtherPars entry:

  pp_def(
    'diagonal',
    OtherPars => 'PDL_Indx whichdims[]',
    MakeComp => '
      if ($COMP(whichdims_count) < 1)
        $CROAK("Diagonal: must have at least 1 dimension");
      /* ... */
    ',
    # ...
  );

There is an XS typemap entry (for PDL_Indx, char*, and pdl* array types for now) that relies on a (varname)_count variable being declared in the XS INPUT section (PP does this for you), to extract the index numbers from an array-ref parameter, and sets the count variable to the right value. PP then makes a copy of the data available. The C function (here, pdl_run_diagonal)'s caller (here, the generated XS function) is responsible for freeing the array passed in (here, PDL's smalloc function is used, so the user need do nothing different).

XS-only OtherPars

As of 2.083, you can prefix the names of OtherPars with $, e.g.

  pp_def('minus',
    OtherPars => 'int $swap',
    # ...
  );

This will mean they are available in HdrCode and FtrCode, but not elsewhere in the generated code (e.g. MakeComp, Code).

The only PP function that we have used in the examples so far is loop. Additionally, there are currently two other functions which are recognised in the Code section:

broadcastloop

As we heard above the signature of a PP defined function defines the dimensions of all the pdl arguments involved in a primitive operation. However, you often call the functions that you defined with PP with pdls that have more dimensions than those specified in the signature. In this case the primitive operation is performed on all subslices of appropriate dimensionality in what is called a broadcast loop (see also overview above and PDL::Indexing). Assuming you have some notion of this concept you will probably appreciate that the operation specified in the code section should be optimised since this is the tightest loop inside a broadcast loop. However, if you revisit the example where we define the pnmout function, you will quickly realise that looking up the IO file descriptor in the inner broadcast loop is not very efficient when writing a pdl with many rows. A better approach would be to look up the IO descriptor once outside the broadcast loop and use its value then inside the tightest broadcast loop. This is exactly where the broadcastloop function comes in handy. Here is an improved definition of pnmout which uses this function:

  pp_def('pnmout',
        Pars => 'a(m)',
        OtherPars => "PerlIO *fp",
        GenericTypes => [qw(B U S L)],
        Code => '
                 int len;
                 len = $SIZE(m) * sizeof($GENERIC());
                 broadcastloop %{
                    if (PerlIO_write($COMP(fp),$P(a),len) != len)
                                $CROAK("Error writing pnm file");
                 %}
  ');

This works as follows. Normally the C code you write inside the Code section is placed inside a broadcast loop (i.e. PP generates the appropriate wrapping C code around it). However, when you explicitly use the broadcastloop function, PDL::PP recognises this and doesn't wrap your code with an additional broadcast loop. This has the effect that code you write outside the broadcast loop is only executed once per transformation and just the code within the surrounding %{ ... %} pair is placed within the tightest broadcast loop. This also comes in handy when you want to perform a decision (or any other code, especially CPU intensive code) only once per thread, i.e.

  pp_addhdr('
    #define RAW 0
    #define ASCII 1
  ');
  pp_def('do_raworascii',
         Pars => 'a(); b(); [o]c()',
         OtherPars => 'int mode',
       Code => ' switch ($COMP(mode)) {
                    case RAW:
                        broadcastloop %{
                            /* do raw stuff */
                        %}
                        break;
                    case ASCII:
                        broadcastloop %{
                            /* do ASCII stuff */
                        %}
                        break;
                    default:
                        $CROAK("unknown mode");
                   }'
   );

As of 2.086, you can put those #defines in a CHeader instead of pp_addhdring so that if you're using multi-C files, it doesn't cause recompilation (or warnings) for any other transformations you pp_def.

types

The types function works similar to the $T macro. However, with the types function the code in the following block (delimited by %{ and %} as usual) is executed for all those cases in which the datatype of the operation is any of the types represented by the letters in the argument to type, e.g.

     Code => '...

             types(BSUL) %{
                 /* do integer type operation */
             %}
             types(FD) %{
                 /* do floating point operation */
             %}
             ...'

You are encouraged to use this idiom (from PDL::Math) in order to minimise effort needed to make your code work with new types:

  use PDL::Types qw(types);
  my @Rtypes = grep $_->real, types();
  my @Ctypes = grep !$_->real, types();
  # ...
    my $got_complex = PDL::Core::Dev::got_complex_version($name, 2);
    my $complex_bit = join "\n",
      map 'types('.$_->ppsym.') %{$'.$c.'() = c'.$name.$_->floatsuffix.'($'.$x.'(),$'.$y.'());%}',
      @Ctypes;
    my $real_bit = join "\n",
      map 'types('.$_->ppsym.') %{$'.$c.'() = '.$name.'($'.$x.'(),$'.$y.'());%}',
      @Rtypes;
    ($got_complex ? $complex_bit : '') . $real_bit;

(although you should first check whether tgmath.h already has a type-generic version of the function you want to call, in which case the above becomes unnecessary).

The RedoDimsCode key is an optional key that is used to compute dimensions of ndarrays at runtime in case the standard rules for computing dimensions from the signature are not sufficient. The contents of the RedoDimsCode entry is interpreted in the same way that the Code section is interpreted-- i.e., PP macros are expanded and the result is interpreted as C code. The purpose of the code is to set the size of some dimensions that appear in the signature. Storage allocation and broadcastloops and so forth will be set up as if the computed dimension had appeared in the signature. In your code, you first compute the desired size of a named dimension in the signature according to your needs and then assign that value to it via the $SIZE() macro.

As an example, consider the following situation. You are interfacing an external library routine that requires an temporary array for workspace to be passed as an argument. Two input data arrays that are passed are p(m) and x(n). The output data array is y(n). The routine requires a workspace array with a length of n+m*m, and you'd like the storage created automatically just like it would be for any ndarray flagged with [t] or [o]. As of 2.088, you can say something like

 pp_def( "myexternalfunc",
  Pars => "p(m); x(n); [o] y; [t] work(wn=CALC($SIZE(n)+$SIZE(m)*$SIZE(m)))",
  ...

If you need to do something more complicated, there is the RedoDimsCode mechanism:

  pp_def(
      "myexternalfunc",
      Pars         => 'p(m); x(n); [o] y(); [t] work(wn);',
      RedoDimsCode => '$SIZE(wn) = $SIZE(n) + $SIZE(m) * $SIZE(m);',
      Code => '
        externalfunc( $P(p), $P(x), $SIZE(m), $SIZE(n), $P(work) );
      '
  );

Note that you can use both =CALC and RedoDimsCode; they don't conflict. As of 2.075, you can use the dimensions of passed-in ndarrays as they are available when the RedoDimsCode is run. Before the code in the Code section is executed PP will create the proper storage for work (one area per POSIX thread, in case of broadcasting that multi-threads - the user cannot supply this). Note that you only took the first dimension of p and x because the user may have sent ndarrays with extra broadcasting dimensions.

You can also use RedoDimsCode to set the dimension of a ndarray flagged with [o]. In this case you set the dimensions for the named dimension in the signature using $SIZE() as in the preceding example.  However, because the ndarray is flagged with [o] instead of [t], broadcasting dimensions will be added if required just as if the size of the dimension were computed from the signature according to the usual rules. Here is an example from PDL::Math

 pp_def("polyroots",
      Pars => 'cr(n); ci(n); [o]rr(m=CALC($SIZE(n)-1)); [o]ri(m);',

The input ndarrays are the real and imaginary parts of complex coefficients of a polynomial. The output ndarrays are real and imaginary parts of the roots. There are n roots to an nth order polynomial and such a polynomial has n+1 coefficients (the zero-th through the nth). In this example, broadcasting will work correctly. That is, the first dimension of the output ndarray with have its dimension adjusted, but other broadcasting dimensions will be assigned just as if there were no RedoDimsCode.

RedoDims passed directly

A RedoDimsCode value as above gets processed, including expanding macros, and adding type-generic loops. For very specific purposes, you may not want this processing done to your dimension-updating code, probably in "slice"-like functions.

Then, instead of passing a RedoDimsCode value, you can pass a RedoDims value (which the RedoDimsCode would otherwise get processed into). Because you will probably want to access the ndarrays, the following macros are provided. They are named assuming you will have the first parameter as PARENT and the second as CHILD, which is the case if you passed a true P2Child value, which you will basically always want to do for this scenario.

RedoDims generated from EquivPDimExpr and EquivDimCheck

Another way to generate the RedoDims code is to supply a EquivPDimExpr and maybe a EquivDimCheck:

  pp_def(
    'xchg',
    OtherPars => 'PDL_Indx n1; PDL_Indx n2;',
    TwoWay => 1,
    P2Child => 1,
    AffinePriv => 1,
    EquivDimCheck => '
      if ($COMP(n1) <0) $COMP(n1) += $PARENT(broadcastids[0]);
      if ($COMP(n2) <0) $COMP(n2) += $PARENT(broadcastids[0]);
      if (PDLMIN($COMP(n1),$COMP(n2)) <0 ||
          PDLMAX($COMP(n1),$COMP(n2)) >= $PARENT(broadcastids[0]))
            $CROAK("One of dims %d, %d out of range: should be 0<=dim<%d",
                $COMP(n1),$COMP(n2),$PARENT(broadcastids[0]));',
    EquivPDimExpr => '
      (($CDIM == $COMP(n1)) ? $COMP(n2) :
       ($CDIM == $COMP(n2)) ? $COMP(n1) :
       $CDIM)
    ',
  );

EquivPDimExpr is evaluated within a loop, and the value of the relevant dimension is available using the macro $CDIM as shown above.

The OtherPars section discussed above is very often absolutely crucial when you interface external libraries with PDL. However in many cases the external libraries either use derived types or pointers of various types.

The standard way to handle this in Perl is to use a typemap file. This is discussed in some detail in perlxs in the standard Perl documentation. In PP the functionality is very similar, so you can create a typemap file in the directory where your PP file resides and when it is built it is automatically read in to figure out the appropriate translation between the C type and Perl's built-in type.

For instance the gsl_spline_init function has the following C  declaration:

    int  gsl_spline_init(gsl_spline * spline,
          const double xa[], const double ya[], size_t size);

Clearly the xa and ya arrays are candidates for being passed in as ndarrays and the size argument is just the length of these ndarrays so that can be handled by the $SIZE() macro in PP. Write an OtherPars declaration of the form

    OtherPars => 'gsl_spline *spl'

and write a short typemap file which handles this type:

    TYPEMAP
    gsl_spline * T_PTR

and use it in the code:

    pp_def('init_meat',
      Pars => 'double x(n); double y(n);',
      OtherPars => 'gsl_spline *spl',
      Code =>'gsl_spline_init,($COMP(spl),$P(x),$P(y),$SIZE(n)));'
    );

where I have removed a macro wrapper call, but that would obscure the discussion.

You can also have OtherPars entries that are "incomplete arrays" of pdl*, both for input and output:

  OtherPars => 'pdl *ins[]', # $COMP(ins_count) will be available
  # OR
  OtherPars => '[o] pdl *outs[]', # update $COMP(outs_count) in your code

Note that the output typemap entry does a free on the array of pdl* pointers, so ensure that you malloc it in your code, without leaking.

OtherPars as outputs

As of 2.081, you can specify an OtherPar as an output. This looks like:

    pp_def('output_op',
      Pars => 'in(n=2)',
      OtherPars => '[o] PDL_Anyval v0; [o] PDL_Anyval v1',
      Code => '
        pdl_datatypes dt = $PDL(in)->datatype;
        ANYVAL_FROM_CTYPE($COMP(v0), dt, $in(n=>0));
        ANYVAL_FROM_CTYPE($COMP(v1), dt, $in(n=>1));
      ',
    );

The passed-in stack SV will be mutated in place, so this code will then work:

    output_op([5,7], my $v0, my $v1);
    is_deeply [$v0,$v1], [5,7], 'output OtherPars work';
    ($v0, $v1) = output_op([5,7]); # you can omit them, then they get returned
    is_deeply [$v0,$v1], [5,7], 'output OtherPars work 1a';

An operation with output OtherPars cannot broadcast, since that would cause undefined results. A runtime check is generated that throws an exception if any Par would cause broadcasting.

Note the syntax for OtherPars has [o] go before the type, while it goes after the type in Pars. It was felt this was the best way to avoid ambiguity given C types can have [] in them.

This relies on the relevant OtherPar having an OUTPUT entry in an XS typemap.

As of 2.083, it is also possible to specify OtherPars as [io], which means they must be supplied (rather than being optional, like an [o] one), but will still be updated after the operation has finished.

You have already heard about the OtherPars key. Currently, there are not many other keys for a data operation that will be useful in normal (whatever that is) PP programming. In fact, it would be interesting to hear about a case where you think you need more than what is provided at the moment. Please speak up on one of the PDL mailing lists. Most other keys recognised by pp_def are only really useful for what we call slice operations (see also above).

One thing that is strongly being planned is variable number of arguments, which will be a little tricky.

An incomplete list of the available keys:

Inplace

Setting this key marks the routine as working inplace - ie the input and output ndarrays are the same. An example is $x->inplace->sqrt() (or sqrt(inplace($x))).

Inplace => 1

Use when the routine is a unary function, such as sqrt.

Inplace => ['a']

If there are more than one input ndarrays, specify the name of the one that can be changed inplace using an array reference.

Inplace => ['a','b']

If there are more than one output ndarray, specify the name of the input ndarray and output ndarray in a 2-element array reference. This probably isn't needed, but left in for completeness.

If bad values are being used, care must be taken to ensure the propagation of the badflag when inplace is being used; consider this excerpt from Basic/Bad/bad.pd:

  pp_def('setbadtoval',HandleBad => 1,
    Pars => 'a(); [o]b();',
    OtherPars => 'double newval',
    Inplace => 1,
    CopyBadStatusCode => 'PDL->propagate_badflag( b, 0 );',
    ...

Since this routine removes all bad values, the output ndarray had its bad flag cleared. This is then propagated to both parents and children.

NOTE: one idea is that the documentation for the routine could be  automatically flagged to indicate that it can be executed inplace, ie something similar to how HandleBad sets BadDoc if it's not  supplied (it's not an ideal solution).

FTypes

  # in slices.pd
  FTypes => {CHILD => '$COMP(totype)'},

The value is a hash-ref mapping parameter-names to an expression giving an override of the type for that parameter. The example above shows the type being overridden to the OtherPars "totype".

OtherParsDefaults

  OtherPars => 'int a; int b',
  OtherParsDefaults => { b => 0 },

Allows specifying default values for OtherPars. It is an error to specify a default for one that is before another that does not have a default.

ArgOrder

  Pars => 'x(); y(); [o]z()'
  OtherPars => 'int a; int b',
  ArgOrder => [qw(x y a b z)],

  # or, a non-reference true value to enable flexible arg-handling and
  # move defaultable to the end, followed by output ndarrays then OtherPars
  Pars => 'x(); y(); [o]z()'
  OtherPars => 'int a; int b',
  ArgOrder => 1,

Allows specifying a different order for providing the operation's arguments. This affects only the generated XS (not C pdl_run_(name)) parameter list; the internal ordering of pdl* in various C arrays is unaffected.

Providing a non-reference true value enables flexible argument-handling and moves defaultable to the end, followed by output ndarrays then output OtherPars. Also, all outputs (ndarray and OtherPars) will be returned on the stack, even if supplied as arguments.

It is an error to specify arguments that are not provided, or to give a false value, or to have "optional" arguments after mandatory ones.

XS argument-handling change

This also changes PP's XS argument handling; normally you can specify:

• just the input/io arguments
• (if the operation has default values provided) those plus values for all arguments with defaults
• all of those plus output arguments, in other words all non-[t] arguments

With ArgOrder given, "optional" arguments (outputs and ones with defaults) will be filled in from the leftmost missing one.

HdrCode

This is C code that is inserted in the XS function before the call to the generated pdl_run_(funcname). It will have access to all the Pars and OtherPars as C values.

FtrCode

As of 2.083. This is C code that is inserted in the XS function after the call to the generated pdl_run_(funcname). It will have access to all the Pars and OtherPars as C values.

So far, we have described the pp_def and pp_done functions. PDL::PP exports a few other functions to aid you in writing concise PDL extension package definitions.

pp_addhdr

Often when you interface library functions as in the above example you have to include additional C include files. Since the XS file is generated by PP we need some means to make PP insert the appropriate include directives in the right place into the generated XS file. To this end there is the pp_addhdr function. This is also the function to use when you want to define some C functions for internal use by some of the XS functions (which are mostly functions defined by pp_def). By including these functions here you make sure that PDL::PP inserts your code before the point where the actual XS module section begins and will therefore be left untouched by xsubpp (cf. perlxs and perlxstut man pages).

A typical call would be

  pp_addhdr('
  #include <unistd.h>       /* we need defs of XXXX */
  #include "libprotos.h"    /* prototypes of library functions */
  #include "mylocaldecs.h"  /* Local decs */

  static void do_the real_work(PDL_Byte * in, PDL_Byte * out, int n)
  {
        /* do some calculations with the data */
  }
  ');

This ensures that all the constants and prototypes you need will be properly included and that you can use the internal functions defined here in the pp_defs, e.g.:

  pp_def('barfoo',
         Pars => ' a(n); [o] b(n)',
         GenericTypes => ['B'],
         Code => ' PDL_Indx ns = $SIZE(n);
                   do_the_real_work($P(a),$P(b),ns);
                 ',
  );

If such things are relevant for only one transformation, consider putting it in a CHeader key for that pp_def (as of 2.086).

pp_addpm

In many cases the actual PP code (meaning the arguments to pp_def calls) is only part of the package you are currently implementing. Often there is additional Perl code and XS code you would normally have written into the pm and XS files which are now automatically generated by PP. So how to get this stuff into those dynamically generated files? Fortunately, there are a couple of functions, generally called pp_addXXX that assist you in doing this.

Let's assume you have additional Perl code that should go into the generated pm-file. This is easily achieved with the pp_addpm command:

   pp_addpm(<<'EOD');

   =head1 NAME

   PDL::Lib::Mylib -- a PDL interface to the Mylib library

   =head1 DESCRIPTION

   This package implements an interface to the Mylib package with full
   broadcasting and indexing support (see L<PDL::Indexing>).

   =cut

   =head2 use_myfunc
        this function applies the myfunc operation to all the
        elements of the input pdl regardless of dimensions
        and returns the sum of the result
   =cut

   sub use_myfunc {
        my $pdl = shift;

        myfunc($pdl->clump(-1),($res=null));

        return $res->sum;
   }

   EOD

pp_add_exported

You have probably got the idea. In some cases you also want to export your additional functions. To avoid getting into trouble with PP which also messes around with the @EXPORT array you just tell PP to add your functions to the list of exported functions:

  pp_add_exported('use_myfunc gethynx');

pp_add_isa

The pp_add_isa command works like the the pp_add_exported function.  The arguments to pp_add_isa are added the @ISA list, e.g.

  pp_add_isa(' Some::Other::Class ');

pp_bless

If your pp_def routines are to be used as object methods use pp_bless to specify the package (i.e. class) to which your pp_defed methods will be added. For example, pp_bless('PDL::MyClass'). The default is PDL if this is omitted.

The value given here (or the default, PDL), anywhere in the .pd file, will be the package into which all PP operations get added, even for operations whose pp_def was called before the pp_bless. This is because that package is inserted at the start of the generated XS code by pp_done. The only way this changes is if pp_addxs is called, which will add the given code (or none if an empty string is given) to the $::PDLPACK package, and then changes the package to the pp_bless value. For historical reasons, this cannot be changed. So, to have several different packages in one .pd file, do something like this:

  # any pp_def up till now will get put in PDL::Pack2
  pp_bless('PDL::Pack1');
  pp_addxs('');
  pp_def('func1', ...);
  pp_bless('PDL::Pack2');
  pp_addxs('');
  pp_def('otherfunc', ...);

pp_addxs

Sometimes you want to add extra XS code of your own (that is generally not involved with any broadcasting/indexing issues but supplies some other functionality you want to access from the Perl side) to the generated XS file, for example

  pp_addxs('','

  # Determine endianness of machine

  int
  isbigendian()
     CODE:
       unsigned short i;
       PDL_Byte *b;

       i = 42; b = (PDL_Byte*) (void*) &i;

       if (*b == 42)
          RETVAL = 0;
       else if (*(b+1) == 42)
          RETVAL = 1;
       else
          croak("Impossible - machine is neither big nor little endian!!\n");
       OUTPUT:
         RETVAL
  ');

Especially pp_add_exported and pp_addxs should be used with care. PP uses PDL::Exporter, hence letting PP export your function means that they get added to the standard list of function exported by default (the list defined by the export tag “:Func”). If you use pp_addxs you shouldn't try to do anything that involves broadcasting or indexing directly. PP is much better at generating the appropriate code from your definitions.

pp_add_boot

Finally, you may want to add some code to the BOOT section of the XS file (if you don't know what that is check perlxs). This is easily done with the pp_add_boot command:

  pp_add_boot(<<EOB);
        descrip = mylib_initialize(KEEP_OPEN);

        if (descrip == NULL)
           croak("Can't initialize library");

        GlobalStruc->descrip = descrip;
        GlobalStruc->maxfiles = 200;
  EOB

pp_export_nothing

By default, PP.pm puts all subs defined using the pp_def function into the output .pm file's EXPORT list. This can create problems if you are creating a subclassed object where you don't want any methods exported. (i.e. the methods will only be called using the $object->method syntax).

For these cases you can call pp_export_nothing() to clear out the export list. Example (At  the end of the .pd file):

  pp_export_nothing();
  pp_done();

pp_core_importList

By default, PP.pm puts the 'use Core;' line into the output .pm file. This imports Core's exported names into the current namespace, which can create  problems if you are over-riding one of Core's methods in the current file. You end up getting messages like "Warning: sub sumover redefined in file subclass.pm" when running the program.

For these cases the pp_core_importList can be used to change what is imported from Core.pm.  For example:

  pp_core_importList('()')

This would result in

  use Core();

being generated in the output .pm file. This would result in no names being imported from Core.pm. Similarly, calling

  pp_core_importList(' qw/ barf /')

would result in

  use Core qw/ barf/;

being generated in the output .pm file. This would result in just 'barf' being imported from Core.pm.

pp_setversion

Simultaneously set the .pm and .xs files' versions, thus avoiding unnecessary version-skew between the two. To use this, simply do this in your .pd file, probably near the top:

 our $VERSION = '0.0.3';
 pp_setversion($VERSION);

 # Then, in your Makefile.PL:
 my @package = qw(FFTW3.pd FFTW3 PDL::FFTW3);
 my %descriptor = pdlpp_stdargs(\@package);
 $descriptor{VERSION_FROM} = 'FFTW3.pd'; # EUMM can parse the format above

However, don't use this if you use Module::Build::PDL. See that module's documentation for details.

pp_deprecate_module

If a particular module is deemed obsolete, this function can be used to mark it as deprecated. This has the effect of emitting a warning when a user tries to use the module. The generated POD for this module also carries a deprecation notice. The replacement module can be passed as an argument like this:

 pp_deprecate_module( infavor => "PDL::NewNonDeprecatedModule" );

Note that function affects only the runtime warning and the POD.

Slice operations need to be able to handle bad values.  The easiest thing to do is look at Basic/Slices/slices.pd to see how this works.

Along with BadCode, there are also the BadBackCode and  BadRedoDimsCode keys for pp_def. However, any EquivCPOffsCode should not need changing, since  any changes are absorbed into the definition of the $EQUIVCPOFFS() macro (i.e. it is handled automatically by PDL::PP).

$variablename_from_sig()

access a pdl (by its name) that was specified in the signature

$COMP(x)

access a value in the private data structure of this transformation (mainly used to use an argument that is specified in the OtherPars section)

$SIZE(n)

replaced at runtime by the actual size of a named dimension (as specified in the signature)

$GENERIC()

replaced by the C type that is equal to the runtime type of the operation

$P(a)

a pointer to the data of the PDL named a in the signature. Useful for interfacing to C functions. Causes that PDL to have a [phys] flag.

$TXYZ(AlternativeX,AlternativeY,AlternativeZ)

expansion alternatives according to runtime type of operation, where XXX is some string that is matched by /[BSULNQFD+]/.

$PDL(a)

return a pointer to the pdl data structure (pdl *) of ndarray a

$ISBAD(a())

returns true if the value stored in a() equals the bad value for this ndarray.  Requires HandleBad being set to 1.

$ISGOOD(a())

returns true if the value stored in a() does not equal the bad value for this ndarray. Requires HandleBad being set to 1.

$SETBAD(a())

Sets a() to equal the bad value for this ndarray. Requires HandleBad being set to 1.

$PRIV()

To access fields in the pdl_trans, eg $PRIV(offs).

$CROAK()

Returns a pdl_error with the supplied (var-args) message, adding the function name at the start, which will cause a barf within the Perl code. This is (as of 2.064) a change in PDL functions' API, so that callers can handle exceptions in their preferred way, which may not use Perl at all.

$EQUIVCPOFFS()

Copy from the PARENT parameter at the first given offset, to the CHILD parameter at the second given offset.

$EQUIVCPTRUNC()

Similar, but if the expression given as the third parameter is false, instead set the CHILD's value to 0.

$DOCOMPALLOC()

Allocates memory for any Comp arrays, after their size has been determined, e.g. here after $COMP(whichdims_count) has been set:

    Comp => 'PDL_Indx whichdims[$COMP(whichdims_count)]',

$DOPRIVALLOC()

As above, except the key is Priv; because it is "Priv", this is only for entries in the pdl_trans itself, and almost certainly only for operations where AffinePriv is true.

$SETNDIMS()

For affine transformations (specifically, ones which set P2Child to true), set the child's ndims to the given value and allocate a suitably-sized array of dimension values.

$SETDIMS()

Similarly for affine transformations, after the above and then the actual dimension sizes are set, use this to resize the child ndarray to the right size.

$SETDELTABROADCASTIDS()

Similarly again, this sets the child's nbroadcastids to the same as the parent's, allocates space for the broadcastids, then sets the child's ones to the same as the parent's plus the given value.

To get a flavour of what broadcastids are for, in the normal way of things the first (0th) one in the parent is the highest dimension-number in it. See PDL::Indexing for more.

loop(DIMS) %{ ... %}

loop over named dimensions; limits are generated automatically by PP

broadcastloop %{ ... %}

enclose following code in a broadcast loop

As of 2.075, threadloop is a deprecated alias for this.

types(TYPES) %{ ... %}

execute following code if type of operation is any of TYPES

PDL::PP's main purpose is to make it easy for you to wrap the broadcasting engine around your own C code, but you can do some other things, too.

pp_def

Used to wrap the broadcasting engine around your C code. Virtually all of this document discusses the use of pp_def.

pp_done

Indicates you are done with PDL::PP and that it should generate its .xs and .pm files based upon the other pp_* functions that you have called. This function takes no arguments.

pp_addxs

This lets you add XS code to your .xs file. This is useful if you want to create Perl-accessible functions that invoke C code but cannot or should not invoke the broadcasting engine. XS is the standard means by which you wrap Perl-accessible C code. You can learn more at perlxs.

pp_add_boot

This function adds whatever string you pass to the XS BOOT section. The BOOT section is C code that gets called by Perl when your module is loaded and is useful for automatic initialization. You can learn more about XS and the BOOT section at perlxs.

pp_addhdr

Adds pure-C code to your XS file. XS files are structured such that pure C code must come before XS specifications. This allows you to specify such C code.

Many functions imported when you use PDL::PP allow you to modify the contents of the generated .pm file. In addition to pp_def and pp_done, the role of these functions is primarily to add code to various parts of your generated .pm file.

pp_addpm

Adds Perl code to the generated .pm file. PDL::PP actually keeps track of three different sections of generated code: the Top, the Middle, and the Bottom. You can add Perl code to the Middle section using the one-argument form, where the argument is the Perl code you want to supply. In the two-argument form, the first argument is an anonymous hash with only one key that specifies where to put the second argument, which is the string that you want to add to the .pm file. The hash is one of these three:

 {At => 'Top'}
 {At => 'Middle'}
 {At => 'Bot'}

For example:

 pp_addpm({At => 'Bot'}, <<POD);
 
 =head1 Some documentation
 
 I know I'm typing this in the middle of my file, but it'll go at
 the bottom.
 
 =cut
 
 POD

Warning: If, in the middle of your .pd file, you put documentation meant for the bottom of your pod, you will thoroughly confuse CPAN. On the other hand, if in the middle of your .pd file, you add some Perl code destined for the bottom or top of your .pm file, you only have yourself to confuse. :-)

pp_beginwrap

Adds BEGIN-block wrapping. Certain declarations can be wrapped in BEGIN blocks, though the default behavior is to have no such wrapping.

pp_addbegin

Sets code to be added to the top of your .pm file, even above code that you specify with pp_addpm({At => 'Top'}, ...). Unlike pp_addpm, calling this overwrites whatever was there before. Generally, you probably shouldn't use it.

When you get compile errors, either from your C-like code or your Perl code, it can help to make those errors back to the line numbers in the source file at which the error occurred.

pp_line_numbers

Takes a line number and a (usually long) string of code. The line number should indicate the line at which the quote begins. This is usually Perl's __LINE__ literal, unless you are using heredocs, in which case it is __LINE__ + 1. The returned string has #line directives interspersed to help the compiler report errors on the proper line.

PDL::PP usually exports all functions generated using pp_def, and usually installs them into the PDL symbol table. However, you can modify this behavior with these functions.

pp_bless

Sets the package (symbol table) to which the XS code is added. The default is PDL, which is generally what you want. If you use the default blessing and you create a function myfunc, then you can do the following:

 $ndarray->myfunc(<args>);
 PDL::myfunc($ndarray, <args>);

On the other hand, if you bless your functions into another package, you cannot invoke them as PDL methods, and must invoke them as:

 MyPackage::myfunc($ndarray, <args>);

Of course, you could always use the PMFunc key to add your function to the PDL symbol table, but why do that?

pp_add_isa

Adds to the list of modules from which your module inherits. The default list is

 qw(PDL::Exporter DynaLoader)

pp_core_importlist

At the top of your generated .pm file is a line that looks like this:

 use PDL::Core;

You can modify that by specifying a string to pp_core_importlist. For example,

 pp_core_importlist('::Blarg');

will result in

 use PDL::Core::Blarg;

You can use this, for example, to add a list of symbols to import from PDL::Core. For example:

 pp_core_importlist(" ':Internal'");

will lead to the following use statement:

 use PDL::Core ':Internal';

pp_setversion

Sets your module's version. The version must be consistent between the .xs and the .pm file, and is used to ensure that your Perl's libraries do not suffer from version skew.

pp_add_exported

Adds to the export list whatever names you give it.  Functions created using pp_def are automatically added to the list. This function is useful if you define any Perl functions using pp_addpm or pp_addxs that you want exported as well.

pp_export_nothing

This resets the list of exported symbols to nothing. This is probably better called pp_export_clear, since you can add exported symbols after calling pp_export_nothing. When called just before calling pp_done, this ensures that your module does not export anything, for example, if you only want programmers to use your functions as methods.