Large Language Models News & Discussions

[bnew]

Spoiler: thread continued

Spoiler: full text & large images

1/3
@rohanpaul_ai
This paper makes complex Multi-objective reinforcement learning (MORL) policies understandable by clustering them based on both behavior and objectives

When AI gives you too many options, this clustering trick saves the day

Original Problem:

Multi-objective reinforcement learning (MORL) generates multiple policies with different trade-offs, but these solution sets are too large and complex for humans to analyze effectively. Decision makers struggle to understand relationships between policy behaviors and their objective outcomes.

-----

Solution in this Paper:

→ Introduces a novel clustering approach that considers both objective space (expected returns) and behavior space (policy actions)

→ Uses Highlights algorithm to capture 5 key states that represent each policy's behavior

→ Applies PAN (Pareto-Set Analysis) clustering to find well-defined clusters in both spaces simultaneously

→ Employs bi-objective evolutionary algorithm to optimize clustering quality across both spaces

-----

Key Insights:

→ First research to tackle MORL solution set explainability

→ Different policies with similar trade-offs can exhibit vastly different behaviors

→ Combining objective and behavior analysis reveals deeper policy insights

→ Makes MORL more practical for real-world applications

-----

Results:

→ Outperformed traditional k-medoids clustering in MO-Highway and MO-Lunar-lander environments

→ Showed comparable performance in MO-Reacher and MO-Minecart scenarios

→ Successfully demonstrated practical application through highway environment case study

2/3
@rohanpaul_ai
Paper Title: "Navigating Trade-offs: Policy Summarization for Multi-Objective Reinforcement Learning"

Generated below podcast on this paper with Google's Illuminate.



https://video.twimg.com/ext_tw_video/1861197491238211584/pu/vid/avc1/1080x1080/56yXAj4Toyxny-Ic.mp4

3/3
@rohanpaul_ai
[2411.04784v1] Navigating Trade-offs: Policy Summarization for Multi-Objective Reinforcement Learning




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[bnew]

Spoiler: thread continued

Spoiler: full text & large images

1/11
@rohanpaul_ai
Transform AI from task-completers to thought-provokers

Current AI primarily act as obedient assistants focused on task completion, stemming from 19th-century statistical models. This limits their potential to enhance human critical thinking and creates a binary perception of AI as either compliant servants or rebellious threats.

-----

Ideas discussed in this Paper:

→ Transform AI from task-completing assistants into provocateurs that challenge users' thinking

→ Implement critical thinking tools from educational frameworks like Bloom's taxonomy and Toulmin model into AI systems

→ Design AI to critique work, surface biases, present counter-arguments, and question assumptions

→ Create interfaces beyond chat that function as "tools of thought" similar to maps, grids, and algebraic notation

2/11
@rohanpaul_ai
Paper Title: "AI Should Challenge, Not Obey"

Generated below podcast on this paper with Google's Illuminate.



https://video.twimg.com/ext_tw_video/1860810218474721280/pu/vid/avc1/1080x1080/mJq53adIVFd2os5n.mp4

3/11
@rohanpaul_ai
[2411.02263] AI Should Challenge, Not Obey



4/11
@jmjjohnson
Sounds like what @arunbahl is building at @AloeInc - a “personal thought partner – a synthetic mind that can reason, purpose-built for human thinking.”



5/11
@rohanpaul_ai
Awesome!!



6/11
@EricFddrsn
That’s great - the people pleasing nature of the LLMs today is one of the main things that separates them of being good thought partners



7/11
@BergelEduardo
Yes! "AI should Challenge, Not Obey." One for Eternity..



8/11
@xone_4
Exactly.



9/11
@HAF_tech
I love this idea! Let's move beyond 19th-century statistical models and create AI that enhances human critical thinking



10/11
@LaneGrooms
Brainstorming: This has been my most successful use case since the first time I ran an LLM locally, I.e. had access to the system prompt. Glad it’s getting more attention.



11/11
@Sipera007
Have you tried randomising all the words in a pdf / source then asking an llm to re order it so it reads as intended? Interesting to see when it breaks vs when it works. For example less than 200 words easy, more than 600 not. Also simply removing all superfluous words like and?




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196