72

Is this why the company wanted to effectively unban AI-generated content? Because AI-generated code explanations will be auto-dumpable now?

From WWC23 - Stack Overflow: Community and AI (Mainstage) at t=25:55, at the section where Prashanth is speaking about the explain code feature of the planned VS Code extension:

Let's say that you had a piece of code you wanted to really get an explanation for the code and you're able to then do that through this mechanism now. You'll get back sort of a short summary of what you're trying to find out or get an explanation for and you're also going to get references from again our data set on where this explanation really was rooted in, right. So this is a really really powerful way for you again to trust and attribute answers from the public community and within stack overflow for teams, and if you think by the way that this was helpful you can create a Q&A, or you can create an article in this case, it's going to suggest tags automatically and you can post it into stack overflow for teams or public stack overflow.

This isn't actually happening as described in the talk, right? The VS Code plugin's explain code feature is not actually going to enable people to dump code explanations from an AI as Q&A pairs to SO, right? You're not going to deploy and promote a feature which blatantly violates site policy, right?

And even if this weren't against site policy, how would it square with community consensus on how code explanation questions should be scoped and formulated? Ex.

I think this would have a pretty severe negative impact on the quality of the knowledge-base over time with content that the poster has dubious understanding of, diluting the value of the knowledge base's content, and filling the internet with AI-generated content that future AI systems will learn bad information from.

And, y'know, having a bunch of your OverflowAI users dumping such Q&A pairs and then getting their questions downvoted and closed for being too broad, and that contributing toward / possibly triggering a question and/or answer ban... isn't really good for your image.

And even if considering all that, you're still going to do this, please at least follow the OpenAI policies for sharing and publication (unlike what you did with your other experiments).

13
  • 1
    related on MSE: Exactly what does "document new learnings and solutions" mean in the description of the Visual Studio Code's OverflowAI extension?
    – starball
    Commented Jul 27, 2023 at 18:53
  • 10
    Ouch... I missed that part with creating content on public Stack Overflow. I don't have enough facepalms.
    – Dalija Prasnikar Mod
    Commented Jul 27, 2023 at 19:04
  • 6
    "So this is a really really powerful way for you again to trust and attribute answers from the public community" - ?????
    – Karl Knechtel
    Commented Jul 27, 2023 at 19:15
  • 5
    I personally, based on my own experience with LLM output, will not trust LLM output for the considerable future. So if the future, is that the company is going to allow and promote the creation of this content, then that output will not be (appreciated, trusted, or considered helpful).
    – Security Hound
    Commented Jul 27, 2023 at 19:21
  • 30
    It's worth mentioning that MDN tried LLM-powered code explanations on their web dev docs sites sites and it did not go well, I think they disabled it after a week or so of bizarre misinformation and angry users
    – user56reinstatemonica8
    Commented Jul 27, 2023 at 19:34
  • 2
    @user56reinstatemonica8 that's not what their own feedback showed though! even gpt4 said their results were great.
    – Kevin B
    Commented Jul 27, 2023 at 19:34
  • 4
    About 20 years ago I worked as code monkey for a group of Comp Sci PH/D candidates working on a system that would parse student code and supply meaningful feedback. The candidates were mostly worried about making the engine parse code into ASTs fast enough to provide real-time feedback. Me, I was wondering how they were planning on building the database needed to interpret the ASTs generated from the student code and who was going to supply the interpretations of often-insane student code.
    – user4581301
    Commented Jul 27, 2023 at 20:33
  • 5
    I made an attempt at copyediting it: "Let's say you have a piece of code you want to get an explanation for. Through this mechanism, you'll get a short summary of what you're trying to learn, including references from our data set on where this summary came from. This is a way to trust and attribute answers from the public community and within Stack Overflow for Teams. If you think the result was helpful, you can create a Q&A pair, or an article; the mechanism will suggest tags automatically, and you can post it into Stack Overflow for Teams or public Stack Overflow."
    – TylerH
    Commented Jul 28, 2023 at 13:38
  • 2
    I’m voting to close this question because, upon further reading, it is a speculative question about future, unpublished site features
    – Tom Wenseleers
    Commented Jul 30, 2023 at 16:08
  • 4
    @TomWenseleers It's phrased as speculation to express disbelief. The transcript of Prashanth's talk is quite clear. Also, asking about the details of planned features being off-topic on meta is news to me, and I don't understand why it would be.
    – starball
    Commented Jul 30, 2023 at 20:49
  • 1
    @starball Well it was new to me too, but my question - very similar to yours - meta.stackoverflow.com/questions/425775/… - was closed for that exact reason...
    – Tom Wenseleers
    Commented Jul 30, 2023 at 21:51
  • @KevinB is it really a "code building" tool? It supposedly explains code. Also, how are they not letting us benefit from it? What would you propose? How would this be useful to you?
    – starball
    Commented Aug 1, 2023 at 17:18
  • @KevinB but... it's supposed to be a VS Code extension? Are we talking about the same thing?
    – starball
    Commented Aug 1, 2023 at 17:29

3 Answers 3

Reset to default
55

I wish the company could again trust the public community and the consensus (read common sense) and just stop pushing for genAI content to be posted on their (our) network.

  • Improving search bar with "AI"? Fine, we can work on that. As long as it does not lead to questions drafted by "AI".

  • Improving question titles (not the body) with "AI"? Sure, it didn't go well the first time, but we can test that further.

  • Letting "AI" hallucinations on the network? No, please no.

  • Dumping unverified "AI" garbage on us? Heck no.

P. S. As Shog9 said (on discord), "First law of holes applies":

If you find yourself in a hole, stop digging

And if you are interested in getting out of there, read his post on MSE.

Why "AI" in quotations? Because LLMs have a long way before they can be called intelligent.

15
  • 1
    i mean... your first two bullets do exactly what 3 and 4 are against.
    – Kevin B
    Commented Jul 27, 2023 at 19:33
  • 2
    @KevinB I can see the issues with the 2nd one (hence testing it further). But how you can make a connection between a search bar and content posted on the site?
    – M--
    Commented Jul 27, 2023 at 19:35
  • A key part of that feature is it turns into a prompt -> answer dialog with a chatbot that will then generate a question for you if you aren't satisfied.
    – Kevin B
    Commented Jul 27, 2023 at 19:41
  • @KevinB where did you get that? I must've missed that part :/
    – M--
    Commented Jul 27, 2023 at 19:43
  • 1
    Image #4 on the announcement and the paragraph above it meta.stackoverflow.com/questions/425766/… (the first three images look like 1...)
    – Kevin B
    Commented Jul 27, 2023 at 19:44
  • @KevinB I don't see that in the images nor the text: i.sstatic.net/f0sOy.jpg
    – M--
    Commented Jul 27, 2023 at 19:46
  • This in particular: 'From here, they’ll see another AI-summarized solution, and as they build their knowledge and possible solutions, they can get more specific in their query, adding in their code or what they’ve tried and making highly personalized search results. These follow-up questions and iterative search experience will help enable users to better understand how the content applies to their context." which is then followed by *"if an instant answer is not available, the user will then engage in a conversation to help them craft a high-quality question"
    – Kevin B
    Commented Jul 27, 2023 at 19:49
  • This process in particular will be able to hallucinate new novel answers that don't exist in the questions returned by the search, particularly once the user is allowed to submit their code. Additionally, this process concludes by generating a question for the user automagically, again, likely to be hallucinated in the same way the formatting assistant did.
    – Kevin B
    Commented Jul 27, 2023 at 19:50
  • 2
    @KevinB I believe you are not reading it objectively and actually reading between the lines; what I read is that they will be prompted into using ask wizard or something like that, not that they will get their question written for them by AI. P.S. putting aside that I said we can work on that feature, not that we'd take whatever we are given.
    – M--
    Commented Jul 27, 2023 at 19:52
  • There's an image floating around from the presentation of a generated question, complete with banner stating something along the lines of "Here's your question ready to be published!" with feedback controls, but I need to step away and couldn't find it quickly.
    – Kevin B
    Commented Jul 27, 2023 at 19:54
  • 2
    @KevinB well I haven't seen that. But that is totally different from what you stated above by quoting and referencing the announcement. P.S. Ask Wizard says something along those lines when you complete writing your question, if I am not mistaken.
    – M--
    Commented Jul 27, 2023 at 19:56
  • 2
    I agree on the first bullet. AI searching can be really useful. Users asking a question usually implies they have limited knowledge on that topic, and hence don't know the exact keywords they need to look for. AI would definitely help.
    – Ricky Mo
    Commented Jul 28, 2023 at 6:26
  • 1
    @M--ßţřịƙïñĝ this is specifically what i was referring to: i.sstatic.net/CYo8U.png -- it's the end result after going through searching, chatting with the chat bot, and then providing your code. It's sourced from the announcement video
    – Kevin B
    Commented Jul 28, 2023 at 19:01
  • @KevinB seen that on discord. Hoping that it was left out of the announcement for a good reason (retracting), not to deliberately misrepresent the feature.
    – M--
    Commented Jul 28, 2023 at 19:04
  • I'm going with it was left out because they won't be ready to release that part in the alpha phase, but who knows
    – Kevin B
    Commented Jul 28, 2023 at 19:06
9

I'm not sure if this is really branded under OverflowAI, but there was a WIRED article about the Google partnership hinting at something like this with Gemini integration. Quoting:

The Google deal will also test how users of the version of Gemini for Google Cloud integration can create new data for Stack Overflow. People who don’t get a satisfactory response from the chatbot will be able to submit their query to Stack Overflow, where once approved by moderators it will be available for the website’s community of users to answer.

This was brought up on MSE in response to Our partnership with Google and commitment to socially responsible AI.

this comment by Rosie says:

if a Gemini user doesn't receive a satisfactory response there, they'll be able to submit their question to Stack Overflow. Like any question on the platform, it needs to meet the same standards and guidelines and can be closed if it does not. The article uses "Moderators" when, in our model, everyone takes moderator actions.

The same section of the WIRED article also says

As the companies prepare for the demo in April, they also are talking about letting users submit improved answers back to Stack Overflow.

So it might not just be questions...

1
-26

I think you'll find genAI-assisted questions would receive a warmer welcome at the language sites---LLMs are, after all, good at (multiple) languages. Hallucinations are rare(r), and more obvious.

Many users of these sites are not native English speakers, which leads to lots of editing; maybe genAI could reduce that workload. Some questions need the OP to add in their attempts or research, point out where they're confused, or explain why Google Translate doesn't answer their question. Some questions are "hit and run" translation requests, which are kind of on-topic but don't attract experts. Maybe genAI can prevent these.

Here's my first AI-assisted questions at Chinese.SE. Here the LLM is functioning as a kind of secretary or typist---it's my question; the LLM wrote down a first draft, and after some back and forth where I gave the AI more details I wanted to include, I edited the result to a final draft and posted it.

(I have a list of questions I want to ask [I jot them down while studying], but I don't have time to convert them to actual questions. Maybe I can finally ask them with genAI's help.)

The main issues I've encountered with AI-assisted question: (a) the LLM making up motivations for asking the question; and (b) the LLM adding in inappropriate phrasing for Stack Exchange (e.g., every sentence is a new question, extended "hellos" and "thank yous", asking for opinions). I tried to get it to add in dictionary references, but that didn't work (it'd need to be done manually). It's also not so great at the Stack Exchange markdown.

Oh, and sometimes the AI actually answers my question in the process of writing.

11
  • 24
    I think this answer is better suited to a similar question on SE's meta... What happens on other SE sites is not very relevant on SO...
    – Cerbrus
    Commented Jul 28, 2023 at 5:37
  • 3
    You're right, but... Stack Exchange have said what happens on SO is going to eventually happen at other sites. And if the feature is not popular here, "why not try another site?" might be a reasonable answer.
    – Rebecca J. Stones
    Commented Jul 28, 2023 at 5:47
  • 5
    "Some questions are "hit and run" translation requests, which are kind of on-topic but don't attract experts." In theory, japanese.SE doesn't allow those.
    – Karl Knechtel
    Commented Jul 28, 2023 at 6:37
  • 7
    Rebecca, developing these AI tools takes considerable resources (wages + time). SO is the core of the network: it gets the vast bulk of the traffic (& hence advertising revenue) and the reputation of SO is what enables SE Inc to sell Teams. If these AI tools work on SO, they may also be extended to other sites, but they simply can't afford to develop them for the minor sites and not SO.
    – PM 2Ring
    Commented Jul 28, 2023 at 6:51
  • I mean they could pioneer these tools somewhere where they'd be more welcome (and are easier to moderate when they go wrong), then bring them over to SO after the issues that arise have been resolved. Things are still fairly experimental. I'm just suggesting the possibility, since the earlier answer is like "we don't want genAI". (Oh, and Chinese.SE is not minor to me.)
    – Rebecca J. Stones
    Commented Jul 28, 2023 at 7:09
  • 12
    Stuff that works on language sites doesn't necessarily work on SO, and vice versa. I don't think the two types of site are comparable at all, and I really don't think implementation experience is transferable like that.
    – Cerbrus
    Commented Jul 28, 2023 at 7:11
  • 5
    @RebeccaJ.Stones Imagine the disaster if this is developed on some language site where it might be more welcome and also could work better simply because LLM are chatbots and they they apply this on technical or other sites where it completely falls apart.
    – Dalija Prasnikar Mod
    Commented Jul 28, 2023 at 7:12
  • 7
    And even with language I have experienced situations where Gen AI translation was totally off the mark as it completely missed the context. I could only catch it because I know both languages well and it was a test. This is rare enough, and it is not as important for asking questions on language sites as others will catch that something is off, but it can be a problem if you allow AI generated answers. It is likely that knowledgeable people would be able to catch errors because sites have low traffic, but imagine what could happen if traffic grows to the point it cannot be handled anymore.
    – Dalija Prasnikar Mod
    Commented Jul 28, 2023 at 7:18
  • 7
    Once you cannot successfully moderate it will be very hard to turn back and prohibit what was previously allowed. If you need proof, just look at SO where at beginning some kinds of questions were allowed, but not anymore, and we still get frequent complaints from people asking similar questions wondering why they are closed and downvoted pointing to existence of such now off topic popular old questions. Same goes for answers. Once you allow AI it is done deal.
    – Dalija Prasnikar Mod
    Commented Jul 28, 2023 at 7:22
  • 5
    @RebeccaJ "they could pioneer these tools somewhere where they'd be more welcome (and are easier to moderate when they go wrong)" Sure, and they have tested some stuff at smaller sites in the past, for those reasons. But that's probably not a useful strategy for these AI tools. They have to work on the scale of SO, which currently has 23.8 million questions, Chinese.SE has ~11,000.
    – PM 2Ring
    Commented Jul 29, 2023 at 17:04
  • 4
    (cont) Also, searching & summarising from a natural language site is a slightly easier task for a language model than searching a mixture of technical natural language and multiple coding languages, where a LLM has a higher risk of hallucinating authoritative-sounding nonsense.
    – PM 2Ring
    Commented Jul 29, 2023 at 17:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .