Learn all of them, choose one, or two, or all of them, or the one your coworker uses, or the one that you think might not break your code in a month, or Python, or DYOR by reading all the other X-vs-Y arguments scattered across the internet already.

Do we have enough moderators to stop all the wars that are going to break out in these discussion forums if its going to be questions like this?

My recommendation is to start with the fundamentals of base R and then switch to Tidyverse at an early stage. Tidyverse functions are usually easier to understand for new users, more readable, and often more performant than base R. The Tidyverse also borrows a number of concepts and function names from other languages, such as SQL, which facilitates venturing beyond R at some point. Just learning key concepts of base R before adopting the Tidyverse is a similar to how many people learn Javascript for front end development: learn just some basic vanilla Javascript and adopt a framework (React, Svelte etc.) at an early stage.

Any casual R user may want to stick with the Tidyverse. I would only deviate from that when exploring more advanced tasks, like package development, web application development, or large data handling.

For package and web application development, it is a good idea to keep the number of dependencies low, corresponding to a heavy use of base R.

Large data tasks make data.table an attractive choice. The syntax and thereby code readability is horrendous, but the performance, in terms of both computational speed and memory efficiency, is definitely better than that of base R and the Tidyverse. There is a Tidyverse wrapper for data.table, but anyone considering to use data.table in the long run might want to use the package's own syntax.

In conclusion, I think that the Tidyverse is the best choice for most R users. For the average user, there is no strong reason to use base R, which is still stuck in the 90s, or data.table, which has an aweful syntax.

What is an "average user"? Is it an undergrad or even high-school student taking their first semester of statistics with no prior programming experience? Perhaps one approach to this could be a tailored list, e.g. "if you are learning statistics for a semester/year, then ______" versus "if you are working with larger data and/or need to scrape as much speed as possible, then data.table" versus something else.

1. they aren’t mutually exclusive. You can use all of them.
2. R is fundamentally fragmented, because it’s so vast. If you’re choosing to pick one of these things to get away from fragmentation you’re probably doing it wrong.
3. if R were re-contrived from being RCPP centric to being LLVM centric that might be pretty surprising opportunity right there. What would it look like if it ran natively on a GPU? LLVM reliably gives O(1000)x speedups versus many libraries. Matlab, python, R. You can also run on a variety of hardware, and tends to adapt to things like core count and memory pretty well.

While learning data analysis everything seems to fit into a data frame. And both data.table and tidyverse are objectively better at handling data frames compared with base R. So it might be tempting to stick with one of those and forget base.

However, in the real world - not everything is a data frame:

• Work with omics? : Bioconductor
• Work with GIS? : R-spatial
• Work with MRI? : Neuroconductor
• Work with heart rate variability? : RHRV.

Not to mention images, graphs, etc. Those things are matrices, arrays, lists, S4 objects. And then knowing base R is unavoidable.

So my best advice would be to get to know some base R, and then, for data frame specific tasks, choose which flavour of data frame manipulation looks most appealing.

A very similar view is expressed here by Brian Caffo.

And also by the creator of data.table - Matt Dowle here.

I do not think it is a fair question as tidyverse includes so much more than just the data manipulation, as it includes ggplot2 and lubridate as well. But for the things data.table can do I wish I had learned right from the start.

I learned tidyverse first with smaller data and all I learned I had to redo after a while when I had to deal with larger data files and more complex manipulation on many columns.

Syntax and readability is really a non issue for me, a larger learning curve maybe but when you get used to it I feel writing data.table is faster ánd more readable. And also more intuitive, I do not have to follow a chain to read that I filter and then perform an action. I rather read on one line / statement that I perform an action on filtered data. If others need to read it, we still can comment code.

Another thing I noticed although stringr in practice is just a wrapper around stringi, on bulk data it seriously comes at a very high cost of processing time (and yes sometimes base can beat it). Also for reading large csv files fread is so much faster.

I hope it makes some sense, beyond code readability but for me it meant rewriting away from tidyverse a reduction from over an hour processing time towards a few minutes for my user cases.

There is no simple answer. There is a compromise, and depending on which factors are most important in each case, or for each user, my recomendation would be different. In addition, personal preference, is a valid reason for choosing among those computer languages or "ecosystems" that are good enough for the job. We are most productive if we feel confortable and enjoy the work we do, so it matters a lot that the tools we use feel good to us. However, one important point is that Tidyverse and data.table are built on top of R, which means that one needs a good understanding of R for debugging problems with either of them. So, starting by properly learning R is worthwhile the effort.

For someone doing data analysis for a living, updates and deprecations maybe tolerable because it is easier to keep up to date with the changes. For the occasional user like many researchers in acatemia or part-time maintainers of R packages, such changes can be a pain. I very much appreciate stabilty and backwards compatibility (I love TeX and LaTeX) but still use the Tidyverse for some data analysis scripts, but rarely pure "Tidyverse". I used data.table for a while some years back, but for the rather small data sets I work with, it did not feel worth the effort of learning it in depth.

On the other hand, trying data.table and learning to use the Tidyverse, even if nowadays I use it rather selectively, has taught me new ways of solving problems and of thinking about data analysis. So, spending time with the three "ecosystems" was time well spent. One thing that I learnt by using R for the last 25 years is that many of the apparent performance bottlenecks in R can be avoided within R itself if one knows how. Performance of R has also improved over the years.

What I wrote above is just my opinion at this time. It is based in how I use these ecosystems. Without actual planning it, my routine has become to use base R unless there is a clear advantage in using the Tidyverse packages and functions. I do not currently use data table, but I would at least try it again if I had to improve performance for large data sets. @Chr I disagree with you about casual users, but mainly because of the constant evolution of the Tidyverse. R is mostly in the 90's but the design is no longer in flux. Again a compromise. @Spacedman I agree with you, I think. These "ecosystems" are tools. Any tool needs to match the task, but also fit the user of the tool.

For my daily work, I use only data.table : faster on heavy dataset, and, as far as I (and my collegues) need it for heavy dataset, the syntax is good also for small ones .. so no need for us to use tidyverse syntax .. base R is, in any case, unavoidable : one needs to know about base R objects in any case. Drawback of this choice : not so easy for us to understand tidyverse syntax on answers in S.O .. :)

I prefer Tidyverse syntax because it's easier to both write and read. I value being able to translate my ideas into code in a quick way, and I also appreciate how I can show Tidyverse code to a non-programmer team mate and most often they'll be able to understand what's going on. I believe writing readable code is really important for data scientists, as we often work with other professionals and social scientists. Therefore, Tidyverse syntax allows us to write readable, understandable and clean code, which are a must in order to avoid giving out the impression that code is some obscure and inaccessible thing to the rest of our team.

Maybe a point to consider about data.table: https://github.com/Rdatatable/data.table/issues/5656

Although this is already a neverending discussion, we should also add polars and arrow to this arena. They are the promising newcomers in the R ecosystem.

Also, there are some packages that join, IMHO, best of both worlds. For instance, dtplyr, tidytable or tidypolars.

I'm with Norm Madoff who has a comprehensive critique.

Basic R operations are important and everybody should be familiar with them. But base R code in connection to data science quickly starts to result in monstrous, unmaintainable code. Think about that: how do you do turn a long data set into a wide data set without pivot_wider?

I have been using only pure base R code for about fifteen years of my life in countless projects. I was very reluctant to start using tidyverse (or even ggplot2) because my learning curve predates these packages. Then I turned to data.table, but I never really liked it. Eventually, a couple of years ago, I grudgingly accepted that coding with pipes and all the goodies that come out of the box in tidyverse packages are really useful and nowadays I can't live without them. I still try to keep tidyverse at minimum (preferably 0) in my own packages, but I do use it a lot in the code I write for scientific projects.

In teaching, I start with basics, so in line with what Norm Matloff writes my students definitely know [, $, logical subscripts and all that. However, we quickly introduce them to basic tidyverse functions for reading and writing files, filtering the data, mutating columns, selecting columns, tidying output of statistical objects, sanitizing data frames and more. We cherry pick the stuff that I consider to be most useful. We teach pipes very late or not at all, but we strongly recommend choosing tibbles over data.frame and tidyverse file reading functions over the builtin functions, because of their consistency and verbosity.

What I really dislike in tidyverse is variable selection. I know the rationale for that, I accept that it makes sense from a certain point of view, but I really hate it. Not least because it is such a pain to code functions around it, and because it puzzles my students.

I try to steer people away from base R when first starting out. I find the tidyverse to be a much easier entry point for new users. I think data.table is slick by being so compact but harder to read out of the gate. There is dplyr compatibility with data.table via dtplyr so users can get the best of both worlds but still I'd start with the tidyverse syntax. I think a lot of stackoverflow answers and tutorials in the wild focus on the tidyverse and that's another reason that I point people that direction.

I am using R for statistics and econometrics. So, for my purpose, I find easystats set of packages more useful than tidyverse.

In my opinion, you need all of them. It's analogous to a triangle; you move towards whichever vertex you require at a given moment. Personally, I prefer data manipulation with tidyverse, but I switch to data.table when speed is a priority. One advantage of using base R is its independence from other packages, especially when creating a package of your own.

The new kid: tidytable !

Allowing us to use dplyr syntax with data.table speed https://www.rdocumentation.org/packages/tidytable/versions/0.10.2

Stack Overflow is a dplyr community. If you ask this question here, you're going to get only one kind of answer.

I admit I don't like dplyr, but that's personal preference. I've managed R and SAS programmers, and a saying in the SAS community that facilitates collaboration is often lost on R programmers: small steps for small minds. When a production programmer gets stumped and needs me to step in to solve issues, it's problematic when their work is buried 10 or 12 levels deep, whether those levels are pipe operands %>% or brackets or parentheses enclosing lapply/apply/sapply loops.

Anyone aspiring to be a professional programmer should do themselves a favor and search job ads in their area. I expect that the posting will be candid about what "flavor" of R they want their programmers to use.

You should learn at least base R and tidyverse as a minimum. data.table is great, because it is fast with great performance, but the syntax is difficult to grasp and the majority of people do not know data.table, but they do tidyverse.

And of course you should know base R. Learning an ecosystem or a framework without understanding the basics of the language does not make sense to me.

Actually the question makes little sense.

Do not forget that tidyverse is not only wraggling packages like dplyr, tidyr or purrr. There are also packages like ggplot2, stringr and lubridate. We can compare all this to base R since base R can also work with strings and dates and can also produce plots. But data.table can't. So obviously you can't use just data.table.

And you can't use only tidyverse either, because there is no tidyverse equivalent for each r base function (there is not set.seed(), mean, rnorm, ...). So no matter how hard you fell in love with tidyverse, you will always use base R language, too.

I've never studied data.table thoroughly, but reading around it seems unquestioned that it is a clear performance winner for large dataset operations.

As for the other two, dplyr and the tidyverse are very terse and readable, but with small to average datasets I noted that base R is almost always much faster than dplyr, up to 50x in my tests.

My guess is as follows:

• Ease of learning and code readability: tidyverse

• Performance, normal datasets (up to hundreds of entries): base R

• Performance, large datasets (thousands of entries): data.table

• Concision: not a huge difference