Philosophy
This proposal is not an object oriented variation on pack and unpack with notions of inherent endianness, read/write head position, or intrinsic codec or charset information. The objects described in this proposal are merely for the storage and direct manipulation of strings and arrays of byte data. Some object oriented conveniences are made, but the exercise of implementing pack, unpack, or an object-oriented analog thereof are left as an exercise for a future proposal of a more abstract type or a ‘struct’ module (as mentioned by Ionut Gabriel Stan on the list). This goes against most mentioned ServerJS/Binary|prior art.
This proposal also does not provide named member functions for any particular subset of the possible charsets, codecs, compression algorithms, or consistent hash digests that might operate on a byte string or array. Instead, convenience member functions are provided for interfacing with any named charset, with the IANA charset name space, and with the possibility of eventually employing a system of modular extensions for other codecs or digests, requiring that the given module exports a specified interface. (As supported originally by Robert Schultz, Davey Waterson, Ross Boucher, and tacitly myself, Kris Kowal, on the First proposition thread on the mailing list). This proposal does not address the need for stream objects to support pipelined codecs and hash digests (mentioned by Tom Robinson and Robert Schultz in the same conversation).
This proposal also reflects both group sentiment and a pragmatic point about properties. This isn’t a decree that properties like “length” should be consistently used throughout the ServerJS APIs. However, given that all engines support properties at the native level (to host String and Array objects) and that byte strings and arrays will require support at the native level, pursuing client-side interoperability is beyond the scope of this proposal and therefore properties have been specified. (See comments by Kris Zyp about the implementability of properties in all engines, comments by Davey Waterson from Aptana about the counter-productivity of attempting to support this API in browsers, and support properties over accessor and mutator functions by Ionut Gabriel Stand and Cameron McCormack on the mailing list).
The byte types provide functions for encoding, decoding, and transcoding, but they are all shallow interfaces that defer to a charset manager module, and may in turn use a system level charset or use a pair of pure JavaScript modules to transcode through an array or stream of canonical Unicode code points. This behavior may be specified further in the future.
Specification
The “binary” top-level module must export “Binary”, “ByteArray” and “ByteString”.
ByteString
A ByteString is an immutable, fixed-width representation of a C unsigned char (byte) array. ByteString supports the String API, and indexing returns a byte substring of length 1.
Constructor
ByteString()
Construct an empty byte string.
ByteString(byteString)
Copies byteString.
ByteString(byteArray)
Use the contents of byteArray.
ByteString(arrayOfNumbers)
Use the numbers in arrayOfNumbers as the bytes. If any element is outside the range 0…255, an exception (”TODO”) is thrown.
ByteString(string, charset)
Convert a string. The ByteString will contain string encoded with charset.
Constructor methods
join(array, delimiter)
Like Array.prototype.join, but for Binarys. Returns a ByteString.
Instance properties
length
The length in bytes. Immutable.
Instance methods (in prototype)
toByteArray()
Returns a byte for byte copy in a ByteArray.
toByteArray(sourceCharset, targetCharset)
Returns a transcoded copy in a ByteArray.
toByteString()
Returns itself, since there’s no need to copy an immutable ByteString.
toByteString(sourceCharset, targetCharset)
Returns a transcoded copy.
toArray()
Returns an array containing the bytes as numbers.
toArray(charset)
Returns an array containing the decoded Unicode code points.
toString()
Returns a debug representation like ”ByteString 10”, where 10 is the length of the Array. Alternative debug representations are valid too, as long as (1) this method will never fail, (2) the length is included.
decodeToString(charset)
Returns the decoded ByteArray as a string.
indexOf(byte)
indexOf(byte, start)
indexOf(byte, start, stop)
Returns the index of the first occurance of byte (a Number or a single element ByteString or ByteArray) or -1 if none was found. If start and/or stop are specified, only elements between the the indexes start and stop are searched.
lastIndexOf(byte)
lastIndexOf(byte, start)
lastIndexOf(byte, start, stop)
Returns the index of the last occurance of byte (a Number or a single element ByteString or ByteArray) or -1 if none was found. If start and/or stop are specified, only elements between the the indexes start and stop are searched.
charCodeAt(offset)
<u>get(offset)</u>
Return the byte at offset as a Number.
byteAt(offset) ByteString
charAt(offset) ByteString
Return the byte at offset as a ByteString.
split(delimiter, [options])
Split at delimiter, which can by a Number, a ByteString, a ByteArray or an Array of the prior (containing multiple delimiters, i.e., “split at any of these delimiters”). Delimiters can have arbitrary size. Options is an optional object parameter with the following optional properties:
- ”count” - Maximum number of elements (ignoring delimiters) to return. The last returned element may contain delimiters.
- ”includeDelimiter” - Whether the delimiter should be included in the result. Returns an array of ByteStrings.
slice()
slice(begin)
slice(begin, end)
See Array.prototype.slice
substr(start)
substr(start, length)
substring(first)
substring(first, last)
[] ByteString
the immutable operator returning ByteStrings
toSource()
which would return “ByteString()” for a null byte string
ByteString does not implement toUpperCase() or toLowerCase() since they are not meaningful without the context of a charset.
ByteArray
A ByteArray is a mutable, flexible representation of a C unsigned char (byte) array.
Constructor
ByteArray()
New, empty ByteArray.
ByteArray(length)
New ByteArray filled with length zero bytes.
ByteArray(byteArray)
Copy byteArray.
ByteArray(byteString)
Copy contents of byteString.
ByteArray(arrayOfBytes)
Use numbers in arrayOfBytes as contents. Throws an exception if any element is outside the range 0…255 (”TODO”).
ByteArray(string, charset)
Create a ByteArray from a Javascript string, the result being encoded with charset.
Unlike the Array, the ByteArray is not variadic so that its initial length constructor is not ambiguous with its copy constructor.
All values within the length of the array are numbers stored as bytes that default to 0 if they have not been explicitly set. Assigning beyond the bounds of a ByteArray implicitly grows the array, just like an Array. Retrieving a value from an index that is out of the bounds of the Array, lower than 0 or at or beyond the length, the returned value is “undefined”. Assigning an index with a value that is larger than fits in a byte will be implicitly and silently masked against 0xFF. Negative numbers will be bit extended to a byte in two’s complement form and likewise masked.
Instance properties
mutable length property
extending a byte array fills the new entries with 0.
Instance methods (in prototype)
toArray()
n array of the byte values
toArray(charset)
an array of the code points, decoded
toString()
A string debug representation like ”ByteArray 10”. Alternative debug representations are valid too, as long as (1) this method will never fail, (2) the length is included.
decodeToString(charset)
returns a String from its decoded bytes in a given charset.
toByteArray()
just a copy
toByteArray(sourceCharset, targetCharset)
transcoded
toByteString()
byte for byte copy
toByteString(sourceCharset, targetCharset)
transcoded
<u>byteAt(offset) ByteString</u>
Return the byte at offset as a ByteString.
<u>get(offset) Number</u>
Return the byte at offset as a Number.
concat(other:ByteArray|ByteString|Array)
pop() byte:Number
push(...variadic Numbers...) -> count(Number)
<u>extendRight(...variadic Numbers / Arrays / ByteArrays / ByteStrings ...)</u>
shift() byte:Number
unshift(...variadic Numbers...) count:Number
<u>extendLeft(...variadic Numbers / Arrays / ByteArrays / ByteStrings ...)</u>
reverse() in place reversal
slice()
sort()
splice()
<u>indexOf()</u>
<u>lastIndexOf()</u>
<u>split()</u>
Returns an array of ByteArrays instead of ByteStrings.
<u>filter()</u>
<u>forEach()</u>
<u>every()</u>
<u>some()</u>
<u>map()</u>
<u>reduce()</u>
<u>reduceRight()</u>
<u>displace(begin, end, values/ByteStrings/ByteArrays/Arrays...) -> length</u>
begin/end are specified like for slice. Can be used like splice but does not return the removed elements.
toSource()
Returns a string like “ByteArray()” for a null byte-array.
[] Number
The mutable operator for numbers
String
The String prototype will be extended with the following members:
toByteArray(charset)
Converts a string to a ByteArray encoded in charset.
toByteString(charset)
Converts a string to a ByteString encoded in charset.
charCodes()
Returns an array of Unicode code points (as numbers).
Array
The Array prototype will be extended with the following members:
toByteArray(charset)
Converts an array of Unicode code points to a ByteArray encoded in charset.
toByteString(charset)
Converts an array of Unicode code points to a ByteString encoded in charset.
General Requirements
None of the specified prototypes or augmentations to existing prototypes are enumerable.
Any operation that requires encoding, decoding, or transcoding among charsets may throw an error if that charset is not supported by the implementation. All implementations MUST support "us-ascii" and "utf-8". Charsets are case insensitive.