Selective deoptimization.

Update the instrumentation to allow selective deoptimization.

Separate instrumentation listener registration from stubs configuration. A
listener is now responsible for configuring the appropriate stubs.
- The method tracing listener installs instrumentation entry/exit stubs or
the interpreter depending on the accuracy of events we want (controlled by
kDeoptimizeForAccurateMethodEntryExitListeners).
- The debugger registers itself as an instrumentation listener but does not
modify methods entrypoints. It only does this on demand when deoptimizing one
method or all the methods.

The selective deoptimization is used for breakpoint only. When a breakpoint is
requested, the debugger deoptimizes this method by setting its entrypoint to
the interpreter stub. As several breakpoints can be set on the same method, we
deoptimize only once. When the last breakpoint on a method is removed, we
reoptimize it by restoring the original entrypoints.

The full deoptimization is used for method entry, method exit and single-step
events. When one of these events is requested, we force eveything to run with
the interpreter (except native and proxy methods). When the last of these
events is removed, we restore all methods entrypoints except those which are
currently deoptimized.

Deoptimizing a method requires all mutator threads be suspended in order to
walk each thread's stack and ensure no code is actually executing while we
modify methods entrypoints. Suspending all the threads requires to not hold
any lock.
In the debugger, we deoptimize/undeoptimize when the JDWP event list changes
(add or remove a breakpoint for instance). During the update, we need to hold
the JDWP event list lock. This means we cannot suspend all the threads at this
time.
In order to deal with these constraints, we support a queue of deoptimization
requests. When an event needs selective/full deoptimization/undeoptimization,
we save its request in the queue. Once we release the JDWP event list lock, we
suspend all the threads, process this queue and finally resume all the threads.
This is done in Dbg::ManageDeoptimization. Note: threads already suspended
before doing this remain suspended so we don't "break" debugger suspensions.

When we deoptimize one method or every method, we need to browse each thread's
stack to install instrumentation exit PC as return PC and save information in
the instrumentation stack frame. Now we can deoptimize multiple times during
the execution of an application, we need to preserve exisiting instrumentation
frames (which is the result of a previous deoptimization). This require to push
new instrumentation frames before existing ones so we don't corrupt the
instrumentation stack frame while walking the stack.

Bug: 11538162
Change-Id: I477142df17edf2dab8ac5d879daacc5c08a67c39

1 files changed, 2 insertions, 0 deletions

diff --git a/runtime/trace.cc b/runtime/trace.cc
index da2c80a..5d053b6 100644
--- a/runtime/trace.cc
+++ b/runtime/trace.cc
@@ -383,6 +383,7 @@ void Trace::Start(const char* trace_filename, int trace_fd, int buffer_size, int
                                                    instrumentation::Instrumentation::kMethodEntered |
                                                    instrumentation::Instrumentation::kMethodExited |
                                                    instrumentation::Instrumentation::kMethodUnwind);
+        runtime->GetInstrumentation()->EnableMethodTracing();
       }
     }
   }
@@ -412,6 +413,7 @@ void Trace::Stop() {
       MutexLock mu(Thread::Current(), *Locks::thread_list_lock_);
       runtime->GetThreadList()->ForEach(ClearThreadStackTraceAndClockBase, NULL);
     } else {
+      runtime->GetInstrumentation()->DisableMethodTracing();
       runtime->GetInstrumentation()->RemoveListener(the_trace,
                                                     instrumentation::Instrumentation::kMethodEntered |
                                                     instrumentation::Instrumentation::kMethodExited |