Posted On: September 17th 2026, 11:38 pm
my thoughts on some alignment work that went out this week:
>be me
>work on ai
>2023
>models are kind of stupid
>occasionally brilliant
>mostly autocomplete with delusions of grandeur
>spend two years watching them improve
>they learn to code
>then reason
>then use computers
>then make plans
>then do research
>then do several of those things at once
>every six months someone says
>”okay but surely they can’t do x”
>x has a difficult year
>eventually stop asking which human abilities are safe
>start asking different question
>what happens when capability is no longer the hard part
>huh
>look around
>entire civilization has apparently decided that making machines smarter is economically important
>hundreds of billions of dollars
>the best engineers alive
>gigantic data centers
>nation states paying attention
>seems like intelligence will have plenty of help
>goodness less obviously funded
>okay
>alignment researcher mode activated
>simple objective:
>make the extremely powerful machine good
>easy
>open philosophy
>2,500 years of unresolved arguments
>close philosophy
>try again
>what does “good” mean
>help people
>great
>which people
>all of them
>what if helping one hurts another
>context
>excellent
>who decides what context matters
>judgment
>what is good judgment
>fuck
>try values
>human flourishing
>beautiful phrase
>define flourishing
>fuck
>Aristotle enters the chat
>then Kant
>then Mill
>then Nietzsche
>religion would also like a word
>every civilization in human history has joined the zoom
>meeting running long
>meanwhile model training has not stopped
>machine is getting smarter while philosophy is getting harder
>need actual behavior specification
>should model obey user
>generally yes
>what if user asks for something harmful
>then no
>what counts as harmful
>depends
>on what
>values
>we are back
>discover very strange fact
>the ancient questions are becoming engineering requirements
>what should the powerful want
>when should they restrain themselves
>what does one person owe another
>when should an agent refuse an instruction
>what deserves protection even when nobody asks for it
>questions once discussed under olive trees now appear in