Scott McKay [Fri, 25 May 2012 14:46:50 +0000 (14:46 +0000)]
Fix a few more things discovered by the tests:
- 'reinitialize-slot' didn't quite work.
- Add a geodata example that uses vectors for repeated fields,
which uncovered a bug in the optimized deserializers.
- Importing the geodata CLOS classes revealed a bug in default
handling when the default is provided only in 'defclass'.
- Fix the knock-on bug in deserialization and the optimized
(de)serialization caused the above.
- Add tighter types in 'decode-uint32' and 'decode-uint64'.
Passes 'precheckin'. Even with the new unit tests in place.
Scott McKay [Wed, 23 May 2012 18:17:11 +0000 (18:17 +0000)]
Add 'vector-of' type, at Shaun Morris's request. This required
some additions to the macrology and to the (de)serialization
code. No runtime performance cost to being able to have things
both ways.
Fix a bug that caused unprocessed options to get dropped.
Improve some of the unit tests.
Passes 'precheckin'. Even with the new unit tests in place.
Scott McKay [Tue, 22 May 2012 20:16:41 +0000 (20:16 +0000)]
Add more stuff to the test suite, and fix what it uncovered.
- Behavior of ':default xxx'/[default=xxx] was not stable,
so add an "empty default" marker and respect it during
parsing and printing.
- 'find-option' now returns a third "found-p" value, useful
for the better handling of defaults.
- Don't print slot readers/writers if they have the same name
as what would be (silently) generated by the macrology.
- Don't print 'import descriptor.proto' lines if there are
no 'lisp_xxx' options in the generated code.
- Add 'clear-field' to the API; add the functions it needs.
- Be more careful about how :conc-name interacts with the
generation of readers/writers.
- When parsing from a .proto file, force the conc-name to
be "" unless requested explicitly not to. This has the
effect of generating readers/writers for all the fields
of imported Protobufs messages, which I think is probably
the desired behavior.
- Add 'make-byte-vector' for convenience.
Passes 'precheckin'. Even with the new unit tests in place.
Scott McKay [Fri, 18 May 2012 21:04:02 +0000 (21:04 +0000)]
Now that Protobufs has a test suite, it found a few things to fix.
- Don't generate warnings for anonymous enums, they're harmless
and ubiquitous.
- 'member' types where are all the members are keywords, is a symbol,
which got converted to a Protobufs 'string'. Wrong.
- If a field has an enum type, the (Lisp) default value should be
treated as a keyword.
- When parsing a .proto file or using the 'define-xxx' macros, any
options that are handled specially should be trimmed from the
options list so that they don't get printed twice.
- Add 'remove-options' for the above.
- Make the options printer be more type-aware.
- Clean up examples.lisp
- 'schemas-equal' can ignore the schema name/class if they're null.
Passes 'precheckin'. Even with the new unit tests in place.
Update the Quake Protobufs schema generator to use the new names.
The Protobufs documentation doesn't say so, but enums can be packed.
- Update the wire-level protocol to (de)serialize packed enums.
- Update the serialization code to use the new wire functions.
Scott McKay [Wed, 16 May 2012 17:01:06 +0000 (17:01 +0000)]
Tie up a few loose ends with 'unittests.proto'; there are still a few
edge case problems, but it's good enough to continue the Stubby work.
The critical .proto files (descriptor.proto, etc) all appear to work.
More precise mapping of (mainly integer) types from .lisp to .proto files
so that we can a truly exact model once we compile the .lisp file. It may
not make any difference, but better safe than sorry.
- Introduce Lisp type 'int32', 'int64', 'uint32', etc
- Make Lisp->Protobufs recognize the new Lisp types
- Make Protobufs schema printers recognize the new types
Fix up the handling of imports and namespaces.
- Always use truenames to record schemas
- Fix typo in 'find-enum' that caused it never to search other schemas
- Record the schema in *all-protobufs* at exactly the right time,
and simplify 'make-load-form' methods
A few tweaks to (de)serialization.
- Make the bodies for empty messages not generate compiler warnings
- Don't make ':packed t' be the default, the user has to say so
- The user documentation doesn't say so, but booleans can be packed.
Scott McKay [Mon, 14 May 2012 23:22:59 +0000 (23:22 +0000)]
A few more model fixes to fully support google/protobuf/unittest.proto
(I love this file, it's got one of everything.)
- The printer should not print extended fields in any message
that further extends an extended message.
- If parsing or importing a .proto file needs a Lisp package that
does not exist, go ahead and create it.
- Importing now tracks all the schemas imported by the "parent",
this so that we can search namespaces thoroughly.
- It looks like 'define-extend' can accept groups. For a deprecated
feature, it sure is a pain in the neck.
- Make 'find-message' and 'find-enum' search all namespaces.
- Print escaped strings, just for completeness.
- Annotate a couple of common options with their types.
Passes 'precheckin'. Passes my by-hand tests and is epsilon away from
handling the worst .proto file around.
Scott McKay [Sun, 13 May 2012 15:17:06 +0000 (15:17 +0000)]
Get google/protobuf/unittest.proto working, insofar as it parses
completely and produces what appears to be a correct model. Next
up, ensure (de)serialization works with this complex beast, which
sets the stage for running the (very complete) Google tests.
- Be much more careful to preserve type information in option
values and field defaults, both in the macros and the parser.
Make the slots in their model objects be untyped.
- This makes it easier for the Proto and Lisp printers to produce
exactly the right output, so fix them, too. Be extra careful
with booleans.
- Fix the printer to recognize more of the standard options so
that their values get printed with the correct type.
- Learn to parse more complicated numeric and integer constants.
- Learn to parse escaped strings.
- Learn to parse comma-separated options.
- Learn to parse extensions with no 'to' specifier.
- Fix the places that used (deprecated) :single instead of :float.
- Proto enum values always map to Lisp keywords, so don't futz
with packages for them.
- Add some more doc strings.
- Add some more examples-that-will-become-tests.
Scott McKay [Fri, 11 May 2012 20:17:48 +0000 (20:17 +0000)]
Now that I've started importing the Google tests for Protos,
a few more bugs are showing up:
- Add basic handling for namespaces
- A typo in my own tests caused me to miss the fact that
negative signed integers did not deserialize correctly
(thanks, Shaun!)
- Just dike out the rest of the Quux dependencies, since it's
pretty clear non-QRes folks will be using it
Scott McKay [Thu, 10 May 2012 21:14:58 +0000 (21:14 +0000)]
Fix some problems reported by Shaun Morris:
- Looks like Stubby will need 'merge-from-array', make room for that.
- Text format deserializer didn't correctly parse negative integers.
- Simply 'deserialize-object' a bit
- Fix a bug in the 'deserialize-object' that caused it not to work
correctly on extended fields.
- Fix a sloppy bug in 'make-tag' that only showed up in SBCL.
- Add a few more examples, soon to be part of a test suite.
Scott McKay [Thu, 10 May 2012 16:13:14 +0000 (16:13 +0000)]
D'oh, I didn't check in 'defgeneric' in the previous commit,
which caused a unit test to fail. Sorry about that.
Make 'make-load-form' for protobuf schemas and messages update
*all-protobufs* and *all-messages*, since 'initialize-instance'
doesn't get called at load time (something I did not know!).
This time it passes 'precheckin --full', and I'll be sure to
commit all the files.
Scott McKay [Tue, 8 May 2012 18:10:35 +0000 (18:10 +0000)]
Fix some corner initialization corner cases.
In particular, uninitialized optional boolean fields
whose Lisp type is (or (member t nil) null) now work.
Implement 'object-initialized-p' and 'slot-initialized-p'
and their Python lookalikes, 'is-initialized' and 'has-field'.
Update the documentation.
Passes 'precheckin --full+', which makes sense since none
of this is called in normal use yet.
Scott McKay [Fri, 4 May 2012 20:00:39 +0000 (20:00 +0000)]
Well, it turns out that the Protobufs 'group' feature,
which has been deprecated for years, is still in wide
use, e.g., in Chubby's bnsresolver.proto
So, implement support for groups:
- Add a model class for it
- Add .proto and .lisp printers
- Add a 'define-group' macro
- Make the .proto parser know how to parse them
Passes 'precheckin --full+', which makes sense since none
of this is used in normal use yet.
Scott McKay [Wed, 25 Apr 2012 17:53:53 +0000 (17:53 +0000)]
Merge $SVN/branches/qres/swm/borgify-1/qres back into the trunk.
Major changes include:
- Simplification of 'define-db-enumeration',
and fixing clients to use simpler name conventions (no more {})
- Removal of unused Quake functionality (courtesy of Alejandro),
including memcached, Oracle O-R record loaders, Quake-OCI c code
- Addition of MySQL schema generator
- Addition of Protobufs support
(see $Q/lisp/quux/protobugs/cl-protobufs.rst)
Passes 'precheckin --full+'. Changes mutually review by swm and asedeno.
Scott McKay [Thu, 5 Apr 2012 16:32:41 +0000 (16:32 +0000)]
A few things for Bill Woods:
- When deserializing the wire-format, seeing a null byte in the tag
position indicates the end of the message.
- Allow the text-format serializer to generate more compact output.
- Add a more useful "list" example that he could use for sending
stuff to/from his sentence understanding system.
Scott McKay [Wed, 21 Mar 2012 18:15:45 +0000 (18:15 +0000)]
Fix a problem in which 'define-message' could define a circular type.
Add a new 'ensure-protobuf' function that ensures that everything in
a schema is well-defined. Makes debugging easier.
Scott McKay [Wed, 21 Mar 2012 16:02:50 +0000 (16:02 +0000)]
Fully implement class/type "override" so that 'proto:define-message'
and 'proto:define-enum' can be used to fully model existing Lisp
classes and types. This will make it much easier to reuse QRes
objects for its Protobufs API.
Scott McKay [Tue, 20 Mar 2012 14:08:20 +0000 (14:08 +0000)]
Fix some things noticed by Igor:
- The default syntax should be "proto2"
- Always emit the syntax in .proto files
- Parse syntax lines (how did I miss this?)
- .proto package names can't contain dash chars; use underscore
projects / cl-protobufs.git / log