The online racing simulator
(Un)Pack Datatype Converions for PHP.
// TYPES : (all multi-byte types are PC style - lowest byte first)
// =====

// char 1-byte character
// byte 1-byte unsigned integer
// word 2-byte unsigned integer
// short 2-byte signed integer
// unsigned 4-byte unsigned integer
// int 4-byte signed integerz
// float 4-byte float

Pack() / Unpack() $format string equivalents.
char is a.
byte is C.
word is v.
short is s.
unsigned is V.
int is l.
float is f.

-- Special --
NULL is x and is 1 byte (8bits) in length. Should be used for any value that is ALWAYS null such as 'Zero' in some packets. In the case of other packets, where you have an int (4 bytes (32bits)) thats always null, you can use x4 to ignore this length of packet.
So what did this achieve that a link to the php manual and a large pointy stick couldn't? It's not like this is a common question.
But a question that I ask my self from time to time. I also know there are quite a few PHP programmers out here that would find this useful. Don't get me wrong tho, the PHP manual is the best place, but still there are some oddities.
yeah thx that is my problem atm.

some data looks "curious" so i think it is a problem while unpacking c / C

dygear do you code php ?

i write atm on a insim class
Yeah, I write in PHP. The LFSWorldSDK is done in PHP, that makes extensive use of the unpack function.
What's the difference between these two :

I unsigned integer (machine dependent size and byte order)
L unsigned long (always 32 bit, machine byte order)

Machine dependant size? For an integer?
Isn't it safer to use L in this case? (because LFS's integers are always 32 bit?)
Or am I missing something?
According to this site, unpacking unsigned ints seems to be problematic in some cases.
I think your right, i will refer this question on c++ pointer, i know more them!

let say i play with a "Integer" address of a pointer on a 32 Bit computer, my real address will be contain into 32 bit from right to left!

If i play with the same code on a 64bit machine, my Real address will contain 64bit from right to left so at the point is the same address, if they where last 32Bit and not first, depend compiler and machine at that point.

i know some platform design will align bit reverse from we read!

on the other hand, "always 32 bit, machine byte order", will have compiler Conversion! so they always keep the right value aligned as you think will be from compiler setting.

I'm not a expert too! so i just reinforce what you think was right! i think same as you!
Quote from AndroidXP :According to this site, unpacking unsigned ints seems to be problematic in some cases.

yeah php's integer...ing is quite crap. Still don't know why they can't do unsigned ints :/ (or at least, on x32 patforms apparently)
Quote from Victor :yeah php's integer...ing is quite crap. Still don't know why they can't do unsigned ints :/ (or at least, on x32 patforms apparently)

Wild guess, because PHP is typeless and it stores it's ints that way, so it's just eaiser to covert into an explicit bit stream this way.
oh well ... I'm currently more interested in my earlier post above though
Quote from Victor :oh well ... I'm currently more interested in my earlier post above though

I'll look into it when I get home. Kinda don't want to look up the educated answer on the iPhone when I am at work, I've been getting calls all day and I could see my self finding the answer then getting a call and forgetting the answer.
I'll probably use L from now on. LFS's data structures are fixed (eg. InSim), so ints in LFS's cases are always 32 bits. It should be platform INdependant. Even though our OS is just 32 bits and it should not matter, maybe in the future it would be 64 bits and I'd be screwed
(afaik anyway - i'm a bit confused atm, because Xeon processors are 64 bit? Though I guess it's the OS that decides what maximum integer size is ..)
i maked some test into PHP, since php is made to facilitate the task of the dev, much of the conversion are background and invisible to us! the documentation is a little vasiant around thoses point!


What i noted from my test, is Depend on the function or operation you do! but number higher then a normal 32Bits INT, will be converted differente way, depend the function call!

And a 64Bits number converting it to INT, can be ok, but you loose 20bits at the end!


here is the test i made! not so big! since! it not my question but yours!
this is on win32 computer!


<?php
$a32BitsRL = str_replace(" ", "", "11111111 11111111 11111111 11111111");
$a64BitsRL = str_replace(" ", "", "11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111");


echo "32bits\r\n";
echo "bindec, return 'uint and after wize convert to int' from:11111111 11111111 11111111 11111111 == ". bindec($a32BitsRL) ." == ".(bindec($a32BitsRL) & 0xFFFFFFFF)."\r\n";
echo "intval, return 'int' but only first byte to 1 from: 11111111 11111111 11111111 11111111 == ". intval($a32BitsRL) ." == ".(intval($a32BitsRL) & 0xFFFFFFFF)."\r\n";


echo "\r\n64Bits\r\n";
//as calc for windows return big int, but thoses bit are missing:
echo bindec($a64BitsRL)."\r\n";
// 64Bit Maximun Integrer == 9223372036854775807
// PHPWin32: 18446744073710000000 //Calc Win32: 18446744073709551615
echo "intval, Take 64Bit and Put Down to 32Bits and (int)".intval($a64BitsRL,10)."\r\n";

?>

Installed freebsd amd64 (it's not just for amd's) on a Xeon box to have a test.

And the results kinda are weird, or at least, they weren't what i expected :

The test script :

<?php 

function makeHexDump ($data) {
    
$len strlen ($data);
    
$output $hex $ascii "";

    for (
$x=0$x<$len$x++) {
        if (
$x && ($x 8) == 0) {
            
$output .= $hex."   ".$ascii."\n";
            
$hex $ascii "";
        }

        
$dec ord ($data{$x});
        
$hex .= substr ("0".dechex ($dec), -2)." ";
        
$ascii .= ($dec 31 && $dec 127) ? $data{$x} : ".";
    }

    if (
$hex != "")
        
$output .= $hex."   ".$ascii;

    return 
$output;
}

function 
testNumber ($Number) {
    
$Unsigned   $Number;
    
$Packed     pack ("I"$Unsigned);
    
$UnPacked   unpack ("I"$Packed);

    echo 
"Unsigned : ".$Unsigned."\n";
    echo 
"Minus 250 : ".($Unsigned 250)."\n";
    echo 
"Packed : ".makeHexDump ($Packed)."\n";
    echo 
"Unpacked : ".$UnPacked[1]."\n\n";
}

testNumber (0xffffffff);
testNumber (0xffffffffffffff);
testNumber (0xffffffffffffffff);

?>

32bit OS result :
Unsigned : 4294967295
Minus 250 : 4294967045
Packed : ff ff ff ff ....
Unpacked : -1

Unsigned : 72057594037928000
Minus 250 : 72057594037928000
Packed : 00 00 00 00 ....
Unpacked : 0

Unsigned : 1.844674407371E+19
Minus 250 : 1.844674407371E+19
Packed : 00 00 00 00 ....
Unpacked : 0

64bit OS result :
Unsigned : 4294967295
Minus 250 : 4294967045
Packed : ff ff ff ff ....
Unpacked : -1

Unsigned : 72057594037927935
Minus 250 : 72057594037927685
Packed : ff ff ff ff ....
Unpacked : -1

Unsigned : 1.84467440737E+19
Minus 250 : 1.84467440737E+19
Packed : 00 00 00 00 ....
Unpacked : 0

I did the Minus 250 to see what happens if you are doing a calculation with the given number. To my big surprise this went just fine on the 32bit OS, for the unsigned 32 bits number that is. I'm really confused about that.
Values with more than 4 bits were totally unreliable on 32bit OS though.

And values with 64 bits seem to crap PHP's pants.

And to get back on topic, the pack function, I used "I" for this test to see if that makes a difference. But as you can see, the hex dump of the Packed values are always just 4 bytes, no matter which OS i run it on and no matter how big the test number is. So I still don't really know what's happening in PHP.

But to make it a bit more confusing, the UNpack function buggers on all the tests! (which was as expected actually)

I ran this on PHP5.2.6 on both computers btw.
I know this is really late in the game, but did anyone submit a bug report to PHP about this subject?
It physically annoys me that this does not work: (Refering to '/lPos[x]/lPos[y]/lPos[z]/' that part of the unpack string.


<?php 
$IS_CPP 
unpack('CSize/CType/CReqI/CZero/lPos[x]/lPos[y]/lPos[z]/SH/SP/SR/CViewPLID/CInGameCam/fFOV/STime/SFlags'$CPP);

print_r($IS_CPP);

?>

That results in this:
Array
(
[Size] => 32
[Type] => 9
[ReqI] => 0
[Zero] => 0
[Pos[x]] => 65536
[Pos[y]] => 131072
[Pos[z]] => 196608
[H] => 1
[P] => 2
[R] => 3
[ViewPLID] => 5
[InGameCam] => 16
[FOV] => 0
[Time] => 8192
[Flags] => 4096
)

I wish it would read into the Pos array like this, as it would make my life would be easier:

Array
(
[Size] => 32
[Type] => 9
[ReqI] => 0
[Zero] => 0
[Pos] => Array
(
[x] => 65536
[y] => 131072
[z] => 196608
)
[H] => 1
[P] => 2
[R] => 3
[ViewPLID] => 5
[InGameCam] => 16
[FOV] => 0
[Time] => 8192
[Flags] => 4096
)

Grr!
Quote from Victor :What's the difference between these two :

I unsigned integer (machine dependent size and byte order)
L unsigned long (always 32 bit, machine byte order)

Machine dependant size? For an integer?
Isn't it safer to use L in this case? (because LFS's integers are always 32 bit?)
Or am I missing something?

To fully answer this question let's look at each and see where we get.

I is unsigned, so that's 1/4th of the way to where we want to be as our datatype is also unsigned or we are reading a datatype that is unsigned. I is of the integer datatype, so that's half way. This is however, well things fall apart for I, as it's a machine dependent size, should PHP be used on a 64bit computer, the function will take up 8 bytes not the 4 we need. To make things worse the byte order is also could be wrong, very strange results that are hard to debug.


L get's points for being unsigned making it 1/4th of the what we need. It's of the long datatype, but that's ok because a long is an integer so half way there. L is a better choice then I due to it always being 32 bits (4 bytes) in length taking us to 3/4th of the way there. But it fails on the last part of ensuring compatibility between all platforms with it's endianess being undefined and up to the processors architecture to dictate.

V to the rescue! V is unsigned (1/4), an long [Read: integer] (2/4), always 32 bit [4 bytes] in length (3/4), and lastly is little endian making it the same endianess as what LFS will be sending (4/4). This makes V a 100% match to this data type.

So, as it turns out we where both wrong. I'm using V (Yes, in it's capital form) as it offers an always unsigned long (int) that's 32bit (4 bytes) in length, and in the little endian (PC Style - lowest byte first) byte order.

I unsigned integer (machine dependent size and byte order)
L unsigned long (always 32 bit, machine byte order)
V unsigned long (always 32 bit, little endian byte order)

I've also updated the list on top of this page to show the most correct answer in the basic types given. Please bare in mind tho, there is no 100% correct solution too all of the data types due to PHP's pack function not offering full support over the whole datatype range with unsigned and signed, and specificity on endianess.

Quote from Victor :I'll probably use L from now on. LFS's data structures are fixed (eg. InSim), so ints in LFS's cases are always 32 bits. It should be platform INdependant.

It's not independent, because PHP runs on multiple architectures. You will have a problem on system that are big endian at least, and will be the only standing problem with type conversion within PHP's pack function.

Quote from Victor :Even though our OS is just 32 bits and it should not matter, maybe in the future it would be 64 bits and I'd be screwed
(afaik anyway - i'm a bit confused atm, because Xeon processors are 64 bit? Though I guess it's the OS that decides what maximum integer size is ..)

It's ok, so long as Live For Speed is a 32 bit only, InSim will only be able to send 32 bits as it being a 32bit app limits the data length it can handle.

Quote from Victor :To my big surprise this went just fine on the 32bit OS, for the unsigned 32 bits number that is. I'm really confused about that.
Values with more than 4 bits were totally unreliable on 32bit OS though.

32bit to 32bit is apples to apples. As for things larger then 4 bytes being unreliable on a 32bit OS, this make sense in the same way Live For Speed being 32bit only not being able to send larger datatypes then that. Everything has to be 64bit to produce a 64bit output. The CPU, the OS, the App.

Quote from Victor :And values with 64 bits seem to crap PHP's pants.

And to get back on topic, the pack function, I used "I" for this test to see if that makes a difference. But as you can see, the hex dump of the Packed values are always just 4 bytes, no matter which OS i run it on and no matter how big the test number is. So I still don't really know what's happening in PHP.

But to make it a bit more confusing, the UNpack function buggers on all the tests! (which was as expected actually)

I ran this on PHP5.2.6 on both computers btw.

Providing that the CPU is 64bit, and the OS is 64bit, then it might be the PHP build that you had. If the build was 32bit, then it's limited to the 32bit length.

Quote from Dygear :I know this is really late in the game, but did anyone submit a bug report to PHP about this subject?

Status: Not a bug. (Not submitted, I just understand this now.)

---


<?php 
php

    
/* Getting System 'Endianess' @ Runtime. */

    # A hex number that may represent 'abyz'
    
$abyz 0x6162797A;

    
// Long Explanation Example:

    # Convert $abyz to a binary string containing 32 bits
    # Do the conversion the way that the system architecture wants to
    
switch (pack ('L'$abyz)) {
        case 
pack ('V'$abyz):
            echo 
'LFS Native Format is System Native Format';
            
# In this case, using L would be fine.
        
break;
        case 
pack ('N'$abyz):
            echo 
'LFS Native Format is NOT System Native Format';
            
# In this case, you must use V to ensure the data is interparted correctly.
        
break;
        default:
            
$endian 'Your system \'endian\' is unknown.'
            
# In this case, you must use V to ensure the data is interparted correctly.
    
}

    
// Mini Example on Runtime Configuraiton of Reading Datatypes.

    # Check to see if the system architechture bit order is compatable with LFSes native bit order.
    # And if it is not, make sure the datatype is read in the correct way.

    
if (pack ('L'$abyz) === pack ('V'$abyz)) {
        
# It's ok to unpack int's with L. (And should be faster)
    
} else {
        
# You must unpack int's with V. (Might be slower)
    
}

?>

It seems there is also no real point in optimizing how the data is packed / unpack based on system architecture, thus you should always use the most correct format for the (un)pack type no matter what system your on.

LFS Native Format is System Native Format
Start Pack Test
Native Format Test
Start Pack Test
Test Took 1.061283826828 Seconds
Start UnPack Test
Test Took 2.2204580307007 Seconds
Non-Native Format Test
Start Pack Test
Test Took 1.0442390441895 Seconds
Start UnPack Test
Test Took 2.2022821903229 Seconds


<?php 
php

    
/* Getting System 'Endianess' @ Runtime. */

    
$abyz 0x6162797A;

    switch (
pack ('L'$abyz))
    {
        case 
pack ('V'$abyz):
            echo 
'LFS Native Format is System Native Format' PHP_EOL;
            
$Native 'V';
            
$Format 'N';
        break;
        case 
pack ('N'$abyz):
            echo 
'LFS Native Format is NOT System Native Format' PHP_EOL;
            
$Native 'N';
            
$Format 'V';
        break;
        default:
            die(
'Your systems \'endian\' is unknown.' PHP_EOL);
    }

    
/* Start Tests */
    
print 'Start Pack Test' PHP_EOL;

    
// Native Format Test
    
print "\tNative Format Test" PHP_EOL;
    
    print 
"\t\tStart Pack Test" PHP_EOL;
    
$testStart microtime(TRUE);
    for (
$i 0$i 1000000$i++)
        
$pack pack($Native$abyz $i);
    
$testTime microtime(TRUE) - $testStart;
    print 
"\t\tTest Took {$testTime} Seconds" PHP_EOL;

    print 
"\t\tStart UnPack Test" PHP_EOL;
    
$testStart microtime(TRUE);
    for (
$i 0$i 1000000$i++)
        
$pack unpack($Native$abyz $i);
    
$testTime microtime(TRUE) - $testStart;
    print 
"\t\tTest Took {$testTime} Seconds" PHP_EOL;

    
// NonNative Format Test
    
print "\tNon-Native Format Test" PHP_EOL;

    print 
"\t\tStart Pack Test" PHP_EOL;
    
$testStart microtime(TRUE);
    for (
$i 0$i 1000000$i++)
        
$pack pack($Format$abyz $i);
    
$testTime microtime(TRUE) - $testStart;
    print 
"\t\tTest Took {$testTime} Seconds" PHP_EOL;

    print 
"\t\tStart UnPack Test" PHP_EOL;
    
$testStart microtime(TRUE);
    for (
$i 0$i 1000000$i++)
        
$pack unpack($Format$abyz $i);
    
$testTime microtime(TRUE) - $testStart;
    print 
"\t\tTest Took {$testTime} Seconds" PHP_EOL;

?>

[edit]
This might have something to do with a Missy Elliott song:


FGED GREDG RDFGDR GSFDG