doc/retrace.md

# R8, Retrace and map file versioning

Programs compiled by R8 are not structurally the same as the input. For example,
names, invokes and line numbers can all change when compiling with R8. The
correspondence between the input and the output is recorded in a mapping file.
The mapping file can be used with retrace to recover the original stack trace
from the stack traces produced by running the R8 compiled program. More
information about R8 and mapping files can be found
[here](https://developer.android.com/studio/build/shrink-code#decode-stack-trace).

## Additional information appended as comments to the file

The format for additional information is encoded as comments with json formatted
data. The data has an ID to disambiguate it from other information.

### Version

The version information states what version the content of the mapping file is
using.

The format of the version information is:
```
# {"id":"com.android.tools.r8.mapping","version":"1.0"}
```
Here `id` must be `com.android.tools.r8.mapping` and `version` is the version of
the mapping file.

The version information applies to content of the file following the position of
the version entry until either the end of the file or another version entry is
present.

If no version information is present, the content is assumed to have version
zero and no additional mapping information except [Source File](#source-file).

When interpreting the mapping file, any additional mapping information
pertaining to a later version should be ignored. In other words, treated as a
normal comment.

Retracing tools supporting this versioning scheme should issue a warning when
given mapping files with versions higher than the version supported by the tool.

- Version 1.0 was introduced by R8 in version 3.1.21

### Source File

The source file information states what source file a class originated from.

The source file information must be placed directly below the class mapping it
pertains to. The format of the source file information is:
```
some.package.Class -> foo.a:
# {"id":"sourceFile","fileName":"R8Test.kt"}
```
Here `id` must be the string `sourceFile` and `fileName` is the source file name
as a string value (in this example `R8Test.kt`).

Note that the `id` of the source file information is unqualified. It is the only
allowed unqualified identifier as it was introduced prior to the versioning
scheme.

### Synthesized (Introduced at version 1.0)

The synthesized information states what parts of the compiled output program are
synthesized by the compiler. A retracing tool should use the synthesized
information to strip out synthesized method frames from retraced stacks.

The synthesized information must be placed directly below the class, field or
method mapping it pertains to. The format of the synthesized information is:
```
# {'id':'com.android.tools.r8.synthesized'}
```
Here `id` must be `com.android.tools.r8.synthesized`. There is no other content.

A class mapping would be:
```
some.package.SomeSynthesizedClass -> x.y.z:
# {'id':'com.android.tools.r8.synthesized'}
```
This specifies that the class `x.y.z` has been synthesized by the compiler and
thus it did not exist in the original input program.

Notice that the left-hand side and right-hand side, here
`some.package.SomeSynthesizedClass` and `x.y.z` respectively, could be any class
name. It is likely that the mapping for synthetics is the identity, but a useful
name can be placed here if possible which legacy retrace tools would use when
retracing.

A field mapping would be:
```
some.package.Class -> x.y.z:
  int someField -> a
  # {'id':'com.android.tools.r8.synthesized'}
```
This specifies that the field `x.y.z.a` has been synthesized by the compiler. As
for classes, since the field is not part of the original input program, the
left- and right-hand side names could be anything. Note that a field can be
synthesized without the class being synthesized.

A method mapping would be:
```
some.package.SomeSynthesizedClass -> x.y.z:
  void someMethod() -> a
  # {'id':'com.android.tools.r8.synthesized'}
```
This specifies that the method `x.y.z.a()` has been synthesized by the compiler.
As for classes, since the method is not part of the original input program, the
left- and right-hand side names could be anything. Note that a method can be
synthesized without the class being synthesized.

For inline frames a mapping would be:
```
some.package.Class -> foo.a:
    4:4:void example.Foo.lambda$main$0():225 -> a
    4:4:void run(example.Foo):2 -> a
    # {'id':'com.android.tools.r8.synthesized'}
    5:5:void example.Foo.lambda$main$1():228 -> a
    5:5:void run(example.Foo):4 -> a
    # {'id':'com.android.tools.r8.synthesized'} <-- redundant
```
This specifies that line 4 in the method `foo.a.a` is in a method that has
been synthesized by the compiler. Since the method is either synthesized or not
any extra synthesized comments will have no effect.

Synthesized information should never be placed on inlined frames:
```
some.package.Class -> foo.a:
4:4:void example.Foo.syntheticThatIsInlined():225 -> a
# {'id':'com.android.tools.r8.synthesized'}
4:4:void run(example.Foo):2 -> a
```
In the above, the mapping information suggests that the inline frame
`example.Foo.syntheticThatIsInlined` should be marked as `synthesized`. However,
since that method was not part of the input program it should not be in the
output mapping information at all.

### RewriteFrame (Introduced at version 2.0)

The RewriteFrame information informs the retrace tool that when retracing a
frame it should rewrite it. The mapping information has the form:

```
  # { id: 'com.android.tools.r8.rewriteFrame', "
      conditions: ['throws(<exceptionDescriptor>)'],
      actions: ['removeInnerFrames(<count>)'] }
```

The format is to specify conditions for when the rule should be applied and then
describe the actions to take. The following conditions exist:

- `throws(<exceptionDescriptor>)`: Will be true if the thrown exception above is
`<exceptionDescriptor>`

Conditions can be combined by adding more items to the list. The semantics of
having more elements in the list is that the conditions are AND'ed together. To
achieve OR one should duplicate the information.

Actions describe what should happen to the retraced frames if the condition
holds. Multiple specified actions will be applied from left to right. The
following actions exist:

- `removeInnerFrames(<count>)`: Will remove the number of frames starting with
inner most frame. It is an error to specify a count higher than all frames.

An example could be to remove an inlined frame if a null-pointer-exception is
thrown:

```
some.Class -> a:
  4:4:void other.Class.inlinee():23:23 -> a
  4:4:void caller(other.Class):7 -> a\n"
  # { id: 'com.android.tools.r8.rewriteFrame', "
      conditions: ['throws(Ljava/lang/NullPointerException;)'],
      actions: ['removeInnerFrames(1)'] }
```

When retracing:
```
Exception in thread "main" java.lang.NullPointerException: ...
  at a.a(:4)
```

It will normally retrace to:
```
Exception in thread "main" java.lang.NullPointerException: ...
  at other.Class.inlinee(Class.java:23)
  at some.Class.caller(Class.java:7)
```

Amending the last mapping with the above inline information instructs the
retracer to discard frames above, resulting in the retrace result:
```
Exception in thread "main" java.lang.NullPointerException: ...
  at some.Class.caller(Class.java:7)
```

The `rewriteFrame` information will only be applied if the line that is being
retraced is directly under the exception line.

### Outline (Introduced at version 2.0)

The outline information can be used by compilers to specify that a method is an
outline. It has the following format:

```
# { 'id':'com.android.tools.r8.outline' }
```

When a retracer retraces a frame that has the outline mapping information it
should carry the reported position to the next frame and use the
`outlineCallsite` to obtain the correct position.

### Outline Call Site (Introduced at version 2.0, updated at 2.2)

A position in an outline can correspond to multiple different positions
depending on the context. The information can be stored in the mapping file with
the following format:

```
# { 'id':'com.android.tools.r8.outlineCallsite',
    'positions': {
        'outline_pos_1': callsite_pos_1,
        'outline_pos_2': callsite_pos_2,
         ...
     },
     'outline': 'outline':'La;a()I'
  }
```

The `outline` key was added in 2.2 and should be the residual descriptor of the
outline.

The retracer should when seeing the `outline` information carry the line number
to the next frame. The position should be rewritten by using the positions map
before using the resulting position for further retracing. Here is an example:

```
# { id: 'com.android.tools.r8.mapping', version: '2.0' }
outline.Class -> a:
  1:2:int outline() -> a
# { 'id':'com.android.tools.r8.outline' }
some.Class -> b:
  1:1:void foo.bar.Baz.qux():42:42 -> s
  4:4:int outlineCaller(int):98:98 -> s
  5:5:int outlineCaller(int):100:100 -> s
  27:27:int outlineCaller(int):0:0 -> s
# { 'id':'com.android.tools.r8.outlineCallsite',
    'positions': { '1': 4, '2': 5 },
    'outline':'La;a()I'}
```

Retracing the following stack trace lines:

```
  at a.a(:1)
  at b.s(:27)
```

Should first retrace the first line and see it is an `outline` and then use
the `outlineCallsite` for `b.s` at position `27` to map the read position `1` to
position `4` and then use that to find the actual mapping, resulting in the
retraced stack:

```
  at some.Class.outlineCaller(Class.java:98)
```

It should be such that for all stack traces, if a retracer ever see an outline
the next obfuscated line should contain `outlineCallSite` information.

### Catch all range for methods with a single unique position

If only a single position is needed for retracing a method correctly one can
skip emitting the position and rely on retrace to retrace correctly. To ensure
compatibility R8 emits a catch-all range `0:65535` as such:

```
0:65535:void foo():33:33 -> a
```

It does not matter if the mapping is an inline frame. Catch all ranges should
never be used for overloads.

### Residual signature (Introduced at 2.2)

The residual signature information was added to mitigate the problem with no
right hand side signature in mapping files. The information should be placed
directly under the first occurrence of a field or method.

```
com.bar -> a:
com.foo -> b:
  com.bar m1(int) -> m2,
  # { id: 'com.android.tools.r8.residualsignature', signature:'(Z)a' }
```

Similar for fields:
```
com.bar -> a:
com.foo -> b:
  com.bar f1 -> f2,
  # { id: 'com.android.tools.r8.residualsignature', signature:'a' }
```

If the residual definition has changed arguments or return type then the
signature should be emitted. The residual signature has no effect on retracing
stack traces but they are necessary when interacting with residual signatures
through the Retrace Api or for composing mapping files.