# Site Experiment: Using AI (GPT3) to summarise Topics **URL:** \ **Category:** Site Stuff šŸ’¾\ **Created:** [February 23, 2023, 12:40pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359 "2023-02-23T12:40:56Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [February 23, 2023, 12:40pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/1 "2023-02-23T12:40:57Z")
So Iā€™ve started an experiment. You may have noticed these appearing: ![image](https://mvertigo.org/uploads/default/original/2X/c/c319e33a16e7a34d2d7145574b2932fccfdea720.png) We are looking at using AI to summarise the first 40 posts on topics. This data is being sent to Open AI. We will only be doing this on public, crawlable Categories which are consumed by Google and Microsoft in any case. Data is not being added to the remote AI model, but it will be used in their macro analytics around usage of the API. The summaries themselves are only shown to people who are logged in, so crawlers will not see them. Thereā€™s a downvoting button if you donā€™t like a summary. Sufficient downvotes (currently \> 2) will prompt the system to retrieve a new summary in the hope it will be a better one. Any concerns or feedback please Reply here. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [February 23, 2023, 12:49pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/2 "2023-02-23T12:49:50Z")
Iā€™m aware of the current impact that people are unable to post New Topics or Replies. We are working on that now. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [February 23, 2023, 1:00pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/3 "2023-02-23T13:00:24Z")
OK that should be resolved. If not we will take another look! ---
**Author:** ![Katharina](https://mvertigo.org/letter_avatar_proxy/v4/letter/k/7993a0/32.png) [@Katharina](https://mvertigo.org/u/Katharina)\ **Post date:** [February 23, 2023, 1:11pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/4 "2023-02-23T13:11:18Z")
I donā€™t like the summary. I have reduced the medication to zero, this is not reflected in the summary, it gives the wrong impression. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [February 23, 2023, 1:12pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/5 "2023-02-23T13:12:21Z")
Yep, I noticed that, letā€™s downvote it! ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [February 23, 2023, 1:15pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/6 "2023-02-23T13:15:07Z")
OK any ~~new reply or~~ additional downvotes to that Topic will cause it to re-retrieve. If this is consistently a problem I will remove this feature. Weā€™ll also add a tooltip to explain the potential pitfalls of the summary. ---
**Author:** ![Katharina](https://mvertigo.org/letter_avatar_proxy/v4/letter/k/7993a0/32.png) [@Katharina](https://mvertigo.org/u/Katharina)\ **Post date:** [February 23, 2023, 1:20pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/7 "2023-02-23T13:20:31Z")
I am not sure whether the summary is useful at all. I gained lots of benefit by reading through each individual post within a topic. If the topic summary does not sound interesting to me, I would not read it, and consequently miss out on all the information in the individual posts. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [February 23, 2023, 1:54pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/8 "2023-02-23T13:54:36Z")
I hear you. Appreciate the feedback. Iā€™m going to leave it up a couple of days or so then run a community poll on keeping it. Iā€™ve added an important Disclaimer to cover your point: _ā€œDisclaimer: this is highly experimental! This summary is generated by a Large Language Model run by [openai.com](http://openai.com). It is very sophisticated and can create convincing, relevant, syntactically correct text but is prone to inaccuracies which can sometimes be misleading. Use it to get a flavour, but read the full Topic for the definite informationā€_ ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [March 17, 2023, 10:37am UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/9 "2023-03-17T10:37:45Z")
OK so they just released ā€œGPT-4ā€ which Iā€™m told is much superior, so Iā€™ve left the summarising system up for now until we can evaluate that. ---
**Author:** ![sunnysideup](https://mvertigo.org/user_avatar/mvertigo.org/sunnysideup/32/2780_2.png) [@sunnysideup](https://mvertigo.org/u/sunnysideup)\ **Post date:** [April 28, 2023, 11:06pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/10 "2023-04-28T23:06:23Z")
I love the intent of the AI and feel it can be useful as a quick overview of the first 40 posts of the topic, as you state it does. Obviously, many topics have massive amounts of posts and information from many different people over the course of years, which can make the topic very time consuming to read from start to finish. It has to be tough for AI (or even a single person) to summarize those 1st 40 posts, especially because many posts contain cross-topics. With time, I feel AI will get even better as newer versions roll out. I hope itā€™s here to stay! ---
**Author:** ![sheepdog\_lord](https://mvertigo.org/letter_avatar_proxy/v4/letter/s/bc79bd/32.png) [@sheepdog\_lord](https://mvertigo.org/u/sheepdog_lord)\ **Post date:** [April 28, 2023, 11:34pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/11 "2023-04-28T23:34:47Z")
it doesnā€™t work for longer threads because GPT3 doesnā€™t have long enough context length ---
**Author:** ![Katharina](https://mvertigo.org/letter_avatar_proxy/v4/letter/k/7993a0/32.png) [@Katharina](https://mvertigo.org/u/Katharina)\ **Post date:** [April 28, 2023, 11:50pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/12 "2023-04-28T23:50:20Z")
It is often incorrect. For example, one time it stated that ā€œKatherina has Meniereā€™s diseaseā€. I definitely donā€™t have Meniereā€™s disease. I am not reading it anymore. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [April 29, 2023, 6:35am UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/13 "2023-04-29T06:35:35Z")
Yeah Open AI is taking its sweet time providing GPT4 access. We will move to that asap. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [April 29, 2023, 6:36am UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/14 "2023-04-29T06:36:56Z")
Yes. Oooops. Hence caveat and the downvoting system. As soon as we get three downvotes a new summary is created, hopefully a better one. It is also regenerated after three additional replies. Hopefully GPT4 will be an improvement. We might also get it to read user bios to avoid this kind of mistake. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [April 29, 2023, 5:30pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/15 "2023-04-29T17:30:14Z")
@Katharina I will try to improve that summary, can you remind me which Topic it is on? ---
**Author:** ![sheepdog\_lord](https://mvertigo.org/letter_avatar_proxy/v4/letter/s/bc79bd/32.png) [@sheepdog\_lord](https://mvertigo.org/u/sheepdog_lord)\ **Post date:** [April 29, 2023, 5:55pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/16 "2023-04-29T17:55:57Z")
I think even with GPT4, there will not be enough context for longer threads. Once the number of words has exceeded the maximum number of tokens that can be fed into the model, the model will essentially ā€œforgetā€ what happened at the beginning of the thread and just summarize the end. This is why on many of the threads, it seems like its only really summarizing the final post ā€“ it ran out of context ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [April 29, 2023, 6:02pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/17 "2023-04-29T18:02:03Z")
Thatā€™s not how it works currently. It simply stops updating the summary after the 40th Post. It never attempts to summarise after that. But your point that context is important carries. It definitely is! ---
**Author:** ![sheepdog\_lord](https://mvertigo.org/letter_avatar_proxy/v4/letter/s/bc79bd/32.png) [@sheepdog\_lord](https://mvertigo.org/u/sheepdog_lord)\ **Post date:** [April 29, 2023, 6:15pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/18 "2023-04-29T18:15:25Z")
oh okay interesting. didnā€™t know that ---
**Author:** ![Katharina](https://mvertigo.org/letter_avatar_proxy/v4/letter/k/7993a0/32.png) [@Katharina](https://mvertigo.org/u/Katharina)\ **Post date:** [April 29, 2023, 11:46pm UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/19 "2023-04-29T23:46:48Z")
I am unable to find it. I think it might have been in the post ā€œNext steps after 50% reduction in symptomsā€. But it is not there anymore. ---
**Author:** ![turnitaround](https://mvertigo.org/user_avatar/mvertigo.org/turnitaround/32/4441_2.png) [@turnitaround](https://mvertigo.org/u/turnitaround)\ **Post date:** [June 14, 2023, 11:41am UTC](https://mvertigo.org/t/site-experiment-using-ai-gpt3-to-summarise-topics/22359/20 "2023-06-14T11:41:52Z")
Weā€™ve just deployed an experimental upgrade to this system that should be able to summarise Topics taking into account now up to the first 160 Posts (thatā€™s an increase of x4!!) ā— šŸŽˆ