Document-wide arguments:

#+property: header-args:R+ :session *R*

Subtree-wide arguments, conveniently added with C-c C-x p:

* Heading
  :PROPERTIES:
  :HEADER-ARGS:R+: :session *R*
  :END:

Block-wide arguments:

#+begin_src R+ :session *R*

Block-wide multi-line arguments:

#+header: :cache yes
#+begin_src R+ :session *R*

(Every line that begins with #+ or * needs escaping with ,.)