| # R8, Retrace and map file versioning | |
| Programs compiled by R8 are not structurally the same as the input. For example, | |
| names, invokes and line numbers can all change when compiling with R8. The | |
| correspondence between the input and the output is recorded in a mapping file. | |
| The mapping file can be used with retrace to recover the original stack trace | |
| from the stack traces produced by running the R8 compiled program. More | |
| information about R8 and mapping files can be found | |
| [here](https://developer.android.com/studio/build/shrink-code#decode-stack-trace). | |
| ## Additional information appended as comments to the file | |
| The format for additional information is encoded as comments with json formatted | |
| data. The data has an ID to disambiguate it from other information. | |
| ### Version | |
| The version information states what version the content of the mapping file is | |
| using. | |
| The format of the version information is: | |
| ``` | |
| # {"id":"com.android.tools.r8.mapping","version":"1.0"} | |
| ``` | |
| Here `id` must be `com.android.tools.r8.mapping` and `version` is the version of | |
| the mapping file. | |
| The version information applies to content of the file following the position of | |
| the version entry until either the end of the file or another version entry is | |
| present. | |
| If no version information is present, the content is assumed to have version | |
| zero and no additional mapping information except [Source File](#source-file). | |
| When interpreting the mapping file, any additional mapping information | |
| pertaining to a later version should be ignored. In other words, treated as a | |
| normal comment. | |
| Retracing tools supporting this versioning scheme should issue a warning when | |
| given mapping files with versions higher than the version supported by the tool. | |
| - Version 1.0 was introduced by R8 in version 3.1.21 | |
| ### Source File | |
| The source file information states what source file a class originated from. | |
| The source file information must be placed directly below the class mapping it | |
| pertains to. The format of the source file information is: | |
| ``` | |
| some.package.Class -> foo.a: | |
| # {"id":"sourceFile","fileName":"R8Test.kt"} | |
| ``` | |
| Here `id` must be the string `sourceFile` and `fileName` is the source file name | |
| as a string value (in this example `R8Test.kt`). | |
| Note that the `id` of the source file information is unqualified. It is the only | |
| allowed unqualified identifier as it was introduced prior to the versioning | |
| scheme. | |
| ### Synthesized (Introduced at version 1.0) | |
| The synthesized information states what parts of the compiled output program are | |
| synthesized by the compiler. A retracing tool should use the synthesized | |
| information to strip out synthesized method frames from retraced stacks. | |
| The synthesized information must be placed directly below the class, field or | |
| method mapping it pertains to. The format of the synthesized information is: | |
| ``` | |
| # {'id':'com.android.tools.r8.synthesized'} | |
| ``` | |
| Here `id` must be `com.android.tools.r8.synthesized`. There is no other content. | |
| A class mapping would be: | |
| ``` | |
| some.package.SomeSynthesizedClass -> x.y.z: | |
| # {'id':'com.android.tools.r8.synthesized'} | |
| ``` | |
| This specifies that the class `x.y.z` has been synthesized by the compiler and | |
| thus it did not exist in the original input program. | |
| Notice that the left-hand side and right-hand side, here | |
| `some.package.SomeSynthesizedClass` and `x.y.z` respectively, could be any class | |
| name. It is likely that the mapping for synthetics is the identity, but a useful | |
| name can be placed here if possible which legacy retrace tools would use when | |
| retracing. | |
| A field mapping would be: | |
| ``` | |
| some.package.Class -> x.y.z: | |
| int someField -> a | |
| # {'id':'com.android.tools.r8.synthesized'} | |
| ``` | |
| This specifies that the field `x.y.z.a` has been synthesized by the compiler. As | |
| for classes, since the field is not part of the original input program, the | |
| left- and right-hand side names could be anything. Note that a field can be | |
| synthesized without the class being synthesized. | |
| A method mapping would be: | |
| ``` | |
| some.package.SomeSynthesizedClass -> x.y.z: | |
| void someMethod() -> a | |
| # {'id':'com.android.tools.r8.synthesized'} | |
| ``` | |
| This specifies that the method `x.y.z.a()` has been synthesized by the compiler. | |
| As for classes, since the method is not part of the original input program, the | |
| left- and right-hand side names could be anything. Note that a method can be | |
| synthesized without the class being synthesized. | |
| For inline frames a mapping would be: | |
| ``` | |
| some.package.Class -> foo.a: | |
| 4:4:void example.Foo.lambda$main$0():225 -> a | |
| 4:4:void run(example.Foo):2 -> a | |
| # {'id':'com.android.tools.r8.synthesized'} | |
| 5:5:void example.Foo.lambda$main$1():228 -> a | |
| 5:5:void run(example.Foo):4 -> a | |
| # {'id':'com.android.tools.r8.synthesized'} <-- redundant | |
| ``` | |
| This specifies that line 4 in the method `foo.a.a` is in a method that has | |
| been synthesized by the compiler. Since the method is either synthesized or not | |
| any extra synthesized comments will have no effect. | |
| Synthesized information should never be placed on inlined frames: | |
| ``` | |
| some.package.Class -> foo.a: | |
| 4:4:void example.Foo.syntheticThatIsInlined():225 -> a | |
| # {'id':'com.android.tools.r8.synthesized'} | |
| 4:4:void run(example.Foo):2 -> a | |
| ``` | |
| In the above, the mapping information suggests that the inline frame | |
| `example.Foo.syntheticThatIsInlined` should be marked as `synthesized`. However, | |
| since that method was not part of the input program it should not be in the | |
| output mapping information at all. | |
| ### RewriteFrame (Introduced at version 2.0) | |
| The RewriteFrame information informs the retrace tool that when retracing a | |
| frame it should rewrite it. The mapping information has the form: | |
| ``` | |
| # { id: 'com.android.tools.r8.rewriteFrame', " | |
| conditions: ['throws(<exceptionDescriptor>)'], | |
| actions: ['removeInnerFrames(<count>)'] } | |
| ``` | |
| The format is to specify conditions for when the rule should be applied and then | |
| describe the actions to take. The following conditions exist: | |
| - `throws(<exceptionDescriptor>)`: Will be true if the thrown exception above is | |
| `<exceptionDescriptor>` | |
| Conditions can be combined by adding more items to the list. The semantics of | |
| having more elements in the list is that the conditions are AND'ed together. To | |
| achieve OR one should duplicate the information. | |
| Actions describe what should happen to the retraced frames if the condition | |
| holds. Multiple specified actions will be applied from left to right. The | |
| following actions exist: | |
| - `removeInnerFrames(<count>)`: Will remove the number of frames starting with | |
| inner most frame. It is an error to specify a count higher than all frames. | |
| An example could be to remove an inlined frame if a null-pointer-exception is | |
| thrown: | |
| ``` | |
| some.Class -> a: | |
| 4:4:void other.Class.inlinee():23:23 -> a | |
| 4:4:void caller(other.Class):7 -> a\n" | |
| # { id: 'com.android.tools.r8.rewriteFrame', " | |
| conditions: ['throws(Ljava/lang/NullPointerException;)'], | |
| actions: ['removeInnerFrames(1)'] } | |
| ``` | |
| When retracing: | |
| ``` | |
| Exception in thread "main" java.lang.NullPointerException: ... | |
| at a.a(:4) | |
| ``` | |
| It will normally retrace to: | |
| ``` | |
| Exception in thread "main" java.lang.NullPointerException: ... | |
| at other.Class.inlinee(Class.java:23) | |
| at some.Class.caller(Class.java:7) | |
| ``` | |
| Amending the last mapping with the above inline information instructs the | |
| retracer to discard frames above, resulting in the retrace result: | |
| ``` | |
| Exception in thread "main" java.lang.NullPointerException: ... | |
| at some.Class.caller(Class.java:7) | |
| ``` | |
| The `rewriteFrame` information will only be applied if the line that is being | |
| retraced is directly under the exception line. | |
| ### Outline (Introduced at version 2.0) | |
| The outline information can be used by compilers to specify that a method is an | |
| outline. It has the following format: | |
| ``` | |
| # { 'id':'com.android.tools.r8.outline' } | |
| ``` | |
| When a retracer retraces a frame that has the outline mapping information it | |
| should carry the reported position to the next frame and use the | |
| `outlineCallsite` to obtain the correct position. | |
| ### Outline Call Site (Introduced at version 2.0, updated at 2.2) | |
| A position in an outline can correspond to multiple different positions | |
| depending on the context. The information can be stored in the mapping file with | |
| the following format: | |
| ``` | |
| # { 'id':'com.android.tools.r8.outlineCallsite', | |
| 'positions': { | |
| 'outline_pos_1': callsite_pos_1, | |
| 'outline_pos_2': callsite_pos_2, | |
| ... | |
| }, | |
| 'outline': 'outline':'La;a()I' | |
| } | |
| ``` | |
| The `outline` key was added in 2.2 and should be the residual descriptor of the | |
| outline. | |
| The retracer should when seeing the `outline` information carry the line number | |
| to the next frame. The position should be rewritten by using the positions map | |
| before using the resulting position for further retracing. Here is an example: | |
| ``` | |
| # { id: 'com.android.tools.r8.mapping', version: '2.0' } | |
| outline.Class -> a: | |
| 1:2:int outline() -> a | |
| # { 'id':'com.android.tools.r8.outline' } | |
| some.Class -> b: | |
| 1:1:void foo.bar.Baz.qux():42:42 -> s | |
| 4:4:int outlineCaller(int):98:98 -> s | |
| 5:5:int outlineCaller(int):100:100 -> s | |
| 27:27:int outlineCaller(int):0:0 -> s | |
| # { 'id':'com.android.tools.r8.outlineCallsite', | |
| 'positions': { '1': 4, '2': 5 }, | |
| 'outline':'La;a()I'} | |
| ``` | |
| Retracing the following stack trace lines: | |
| ``` | |
| at a.a(:1) | |
| at b.s(:27) | |
| ``` | |
| Should first retrace the first line and see it is an `outline` and then use | |
| the `outlineCallsite` for `b.s` at position `27` to map the read position `1` to | |
| position `4` and then use that to find the actual mapping, resulting in the | |
| retraced stack: | |
| ``` | |
| at some.Class.outlineCaller(Class.java:98) | |
| ``` | |
| It should be such that for all stack traces, if a retracer ever see an outline | |
| the next obfuscated line should contain `outlineCallSite` information. | |
| ### Catch all range for methods with a single unique position | |
| If only a single position is needed for retracing a method correctly one can | |
| skip emitting the position and rely on retrace to retrace correctly. To ensure | |
| compatibility R8 emits a catch-all range `0:65535` as such: | |
| ``` | |
| 0:65535:void foo():33:33 -> a | |
| ``` | |
| It does not matter if the mapping is an inline frame. Catch all ranges should | |
| never be used for overloads. | |
| ### Residual signature (Introduced at 2.2) | |
| The residual signature information was added to mitigate the problem with no | |
| right hand side signature in mapping files. The information should be placed | |
| directly under the first occurrence of a field or method. | |
| ``` | |
| com.bar -> a: | |
| com.foo -> b: | |
| com.bar m1(int) -> m2, | |
| # { id: 'com.android.tools.r8.residualsignature', signature:'(Z)a' } | |
| ``` | |
| Similar for fields: | |
| ``` | |
| com.bar -> a: | |
| com.foo -> b: | |
| com.bar f1 -> f2, | |
| # { id: 'com.android.tools.r8.residualsignature', signature:'a' } | |
| ``` | |
| If the residual definition has changed arguments or return type then the | |
| signature should be emitted. The residual signature has no effect on retracing | |
| stack traces but they are necessary when interacting with residual signatures | |
| through the Retrace Api or for composing mapping files. |