Lrlex codegen - #661
Conversation
| } | ||
| } | ||
|
|
||
| pub(crate) enum LexerBuildEnvError {} |
There was a problem hiding this comment.
I had forgotten to double check this, for some reason I thought a variant had been added to this,
but that was LexerCodegenError.
| ) | ||
| let code_gen = build_env | ||
| .code_generator(self.rule_ids_map, env!("VERGEN_BUILD_TIMESTAMP")) | ||
| .map_err(|e| match e {})?; |
There was a problem hiding this comment.
This wacky map_err/match was necessary at the time because ? couldn't convert the error.
I figured by the time I was done we'd have a variant added. Although I must admit I'm a bit uncertain what the type returned is, I assume it's inferring ! or some such.
| pub(crate) fn gen_token_consts(&self) -> TokenStream { | ||
| let mut token_consts = TokenStream::new(); | ||
| if let Some(rim) = self.rule_ids_map() { | ||
| let mut rim_sorted = Vec::from_iter(rim.iter()); |
There was a problem hiding this comment.
This thing just emits an empty TokenStream if rule_ids_map is None, I assume that is okay?
Meaning something like lrpar didn't run? This copies over the existing code, but I was curious whether this needs to be as-is, or if it could/should be checked like an error condition?
There was a problem hiding this comment.
For some reason I forgot we had lrlex/examples, sure enough if we add assert!(self.rules_id_map.is_some()), the example lrlex/examples/calclex fails. So this seems correct as is.
2 participants
This is an attempt at lrlex codegen, in the style of the recently added lrpar codegen module.
One thing this doesn't do is change the
CTTokenMapBuilder.This tries to sequence the move over a couple of commits, with the first commit just generating module and ownership structure. With each subsequent commit moving one aspect of code generation, until it's all in the codegen module.
One slightly suprising thing was the placement of lexerdef within the
LexerBuildEnv, because we need it to generate therule_ids_map, with that we can wait to generate theLexerCodegenutil therule_ids_mapis built, and pass it in to thecode_generatorfunction, this is a slight departure from thelrparapi, which merely took atimestamp.Edit: One thing that the additional
rule_ids_mapparameter I immediately notice is that unlikeBuildEnvArgsand other builder patterns, it's hard for us to add add an additional parameter should we need one. I wonder if we should considerLexerCodegenArgs?