-
For whatever reason, I have never had the brain for mold making; any kind of intuitive understanding of the process alludes me. When to use a two part mold, what objects are even suitable for casting, etc. Despite all of this, I periodically get the itch to try it again which is exactly what I did this weekend.
I ordered some jesmonite, an interesting cross between plaster and resin that is really difficult to find in the United States despite being quite popular in the U.K, and decided to try casting a sycamore tree seed and two decorative gourds I grew last summer.
I was completely unable to remove the sycamore seed from the silicone mold. It was probably too rough and porous. Next time I’ll try using some sort of mold release.
The two gourds came out great though! Afterwards, I tried painting them with watercolors which worked much better than I was expecting it to.
-
§ No work next Monday for Martin Luther King Day and then a “work from home” faculty work day on Tuesday. Great. On Wednesday, students will rotate classes for their third quarter which means I’ll be teaching a group of kids I haven’t seen in eight-ish weeks. I expect it will be a nice change of pace.
§ I started watching Three Pines which honestly hasn’t hooked me yet and mostly had the effect of making me want to re-watch Twin Peaks.
I also saw The Devil’s Hour. I thought it was pretty good and I was super happy to see that it’s a limited series. It turns out, stories are often better when they have a pre-planned beginning, middle, and end. Perhaps the accelerated rate that streaming services are canceling show renewals will encourage this trend to continue.
Finally, I saw Pearl, Ti West’s prequel to X. I thought it had a fantastic atmosphere. The music was great, the set design had a fascinating quality of period authenticity while at the same time being unsettlingly plastic, even the colors were interesting in a way I can’t exactly place.
§ I swear, at some point in the past ten years autumn disappeared. The Midwest seemingly now transitions from 85 °F to 35 °F overnight. The season must have been more distinct before; whenever asked I would always list it as my favorite! Anyway, there were a few days in the 50s this week which was nice. Although, on balance, we also had like four inches of snow on Friday.
I’ve noticed that the days getting longer is giving me an unexpected optimism. I am already starting to think about which vegetables I would like to try growing in the spring.
§ Links
- Analog chess
- “This is a version of chess where the pieces are not constrained to an 8x8 grid, and instead can move to any position on the board.”
- See also: Really Bad Chess
- Giffusion — Create GIFs using Stable Diffusion
- I tried it out on Google Collab. It was a bunch of fun but the results weren’t especially impressive. I am still super excited for a true generative model for animation.
- GLM-130B is an open source large language model. However, you should proceed with caution.
- Q&A against documentation with GPT3 + OpenAI embeddings
- A method of prompt engineering to easily “fine tune” GPT3 on your own data
§ Recipes
- Gamja-tang — Korean pork and potato stew
- Hmm… this recipe was good but 1) it tastes surprisingly similar to Kapusniak while 2) requiring a significantly more involved process to cook. I will probably make it again sometime though!
- Chana masala
- One of my favorites. Plus this used a bunch of frozen tomatoes from the garden, freeing up space in the freezer.
- Not a recipe but I ate a pomelo — the largest citrus fruit — for the first time. I am tempted to say that I think it might be better than grapefruit. Much less bitter and possibly slightly sweater.
- Analog chess
-
Eleven Labs recently shared a demo of their new voice synthesis AI. It is worth listening to the audio samples. While I don’t think they are significantly better than the recent demo released by Apple, it is for precisely that reason that I think this is so noteworthy — the fact that small companies are able to build comparable offerings to the industry’s largest players is impressive.
Also, I have to admit, their Steve Jobs voice simulation demo is impressive.
Finally, as time goes on I am increasingly unable to understand why none of these recent advancements have trickled down into voice assistants. Why not hook up a speech recognition AI to GPT and then speak the result using one of these voice generation AIs? It must be inference cost, right? Otherwise, I must be missing something.
Microsoft and OpenAI together could use Whisper, to ChatGPT, to VALL-E and dub it Cortana 2.0. Or put it in a smart speaker and instantly blow Amazon Alexa, Apple Homepod, Google Home offerings out of the water. And that is just using projects OpenAI and Microsoft released publicly!
-
I wrote in December about how ChatGPT could be improved by routing relevant questions to Wolfram Alpha — i.e. neuro-symbolic AI. It sounds like Stephen Wolfram has similar thoughts:
There’ll be plenty of cases where “raw ChatGPT” can help with people’s writing, make suggestions, or generate text that’s useful for various kinds of documents or interactions. But when it comes to setting up things that have to be perfect, machine learning just isn’t the way to do it—much as humans aren’t either.
[…]
ChatGPT does great at the “human-like parts”, where there isn’t a precise “right answer”. But when it’s “put on the spot” for something precise, it often falls down. But the whole point here is that there’s a great way to solve this problem—by connecting ChatGPT to Wolfram|Alpha and all its computational knowledge “superpowers”.
[…]
Inside Wolfram|Alpha, everything is being turned into computational language, and into precise Wolfram Language code, that at some level has to be “perfect” to be reliably useful. But the crucial point is that ChatGPT doesn’t have to generate this. It can produce its usual natural language, and then Wolfram|Alpha can use its natural language understanding capabilities to translate that natural language into precise Wolfram Language.
These are exactly the types of informal integrations I expect to see in spades once we finally get a viable open source alternative to GPT.
-
Microsoft has been in talks to invest $10 billion into the owner of ChatGPT… The funding, which would also include other venture firms, would value OpenAI… at $29 billion, including the new investment
Whether you think $29 billion is a sensible valuation for OpenAI depends a lot of what you think of their future… On [one] hand, being valued at $29 billion dollars is really a lot for an AI company, historically speaking, on the other Altman often publicly hints that the company is close to AGI
How much would AGI actually be worth? A few years back, PwC estimated that the overall AI market might be worth over $15 Trillion/year by the year 2030; McKinsey published a similar study, coming at at $13 trillion/year… If you really were close to being first to AGI, wouldn’t you want to stick around and take a big slice of that, with as much control as possible? My best guess? Altman doesn’t really know how to make OpenAI into the juggernaut that everybody else seems to think he’s got.
Finally, Marcus shares some interesting information he received from an anonymous source:
Turns out Semafor was wrong about the deal terms. If things get really really good OpenAI gets back control; I am told by a source who has seen the documents “Once $92 billion in profit plus $13 billion in initial investment are repaid [to Microsoft] and once the other venture investors earn $150 billion, all of the equity reverts back to OpenAI.” In that light, Altman’s play seems more like a hedge than a firesale; some cash now, a lot later if they are hugely successful.
It is important to remember that OpenAI isn’t exactly a for-profit company but, instead, a “capped profit” company. From their press release announcing the new corporate structure:
The fundamental idea of OpenAI LP is that investors and employees can get a capped return if we succeed at our mission… But any returns beyond that amount… are owned by the original OpenAI Nonprofit entity.
OpenAI LP’s primary fiduciary obligation is to advance the aims of the OpenAI Charter, and the company is controlled by OpenAI Nonprofit’s board. All investors and employees sign agreements that OpenAI LP’s obligation to the Charter always comes first, even at the expense of some or all of their financial stake.
Although at the end of the day OpenAI can always change course. From The Information:
OpenAI has proposed a key concession as part of discussions with potential new investors. Instead of putting a hard cap on the profit sharing—essentially their return on investment—it could increase the cap 20% per year starting around 2025, said a person briefed on the change. Investors say this compromise, if it goes through, would make the deal more attractive because it would allow shareholders to obtain venture-level returns if the company becomes a moneymaker.
-
LAION-AI, the non-profit organization that created the original dataset behind Stable Diffusion, launched Open Assistant last week. From the project’s GitHub page:
Open Assistant is a project meant to give everyone access to a great chat based large language model… In the same way that stable-diffusion helped the world make art and images in new ways we hope Open Assistant can help improve the world by improving language itself.
Remember, LAION is not the company behind Stable Diffusion (that would be Stability AI), they just produced the training dataset. We have yet to see if they can build a successful product. They have genuinely exciting plans though!
We are not going to stop at replicating ChatGPT. We want to build the assistant of the future, able to not only write email and cover letters, but do meaningful work, use APIs, dynamically research information, and much more, with the ability to be personalized and extended by anyone. And we want to do this in a way that is open and accessible, which means we must not only build a great assistant, but also make it small and efficient enough to run on consumer hardware.
Whether or not LAION is able to accomplish their goals, I am optimistic that we will see serious developments in the open source large language model space this year.
-
John Naughton, writing for The Guardian:
[ChatGPT] reminds me, oddly enough, of spreadsheet software, which struck the business world like a thunderbolt in 1979 when Dan Bricklin and Bob Frankston wrote VisiCalc, the first spreadsheet program, for the Apple II computer
Eventually, Microsoft wrote its own version and called it Excel, which now runs on every machine in every office in the developed world. It went from being an intriguing but useful augmentation of human capabilities to being a mundane accessory
Digital spreadsheets are perhaps the best example of a computational tool successfully augmenting the day-to-day work of a huge number of people. Spreadsheets have gone from nonexistent to simultaneously indispensable and mundane unbelievably quickly. If a similar augmentation occurs for prose it will be an equally, if not more, transformative development.
-
Jeff Kaufman wrote a fascinating piece arguing that nearly all online advertisement is probably illegal under GDPR as it currently stands:
I think the online ads ecosystem is most likely illegal in Europe, and as more decisions come out it will become clear that it can’t be reworked to be within the bounds of the GDPR.
The most surprising thing I learned from this article is that apparently it is legally required that cookie consent banners make the process of opting out as easy as opting in. I don’t think I have ever encountered a site where that is the case.
-
§ Back to teaching after two weeks of winter vacation. Although, as always, I wish the vacation was longer, it feels nice to start getting back into my normal routines after the crazy holiday season. Worst case scenario: ten weeks until spring break, twenty-one until summer.
§ I have been listening to the album Distance by the band Erasers a lot after discovering it on James Reeves' list of favorite albums of 2022. Overall, the list is full of great minimal electronic artists that are all new to me. It is going to make the perfect soundtrack for some gray winter days ahead.
§ Longmont Potion Castle 20 was released on Friday. The tracks I have had the opportunity to listen to so far are amazing, as usual.
§ Three of the quails escaped into the garage which made for a real Yakety Sax evening as Caroline and I ran around trying to catch them in makeshift nets.
§ Links
- Underground greenhouse (Wallapini)
- Interactive Illustrations from Code: The Hidden Language of Computer Hardware and Software
- Paper programs
- See also: Dynamicland
§ Recipes
Getting back into my work schedule this week meant much less cooking at home. I did at least get the opportunity to make one new-to-me recipe — arroz con pollo.
Recipe discovery is difficult. I would love to find a personal cooking blog that is not full of SEO spam.
- Cajun sausage and rice skillet
- An old classic. I had to use some kielbasa that was left over from Kapusniak last week. Easy and quick to make and goes great with cornbread.
- Arroz con pollo
- This was good but not quite as good as my favorite Spanish rice recipe. I will definitely incorporate some elements from that recipe if I make this one again. A big positive is that I now have a huge quantity of very versatile leftovers.
-
Ann Gibbons, writing for Science.org:
Ask medieval historian Michael McCormick what year was the worst to be alive, and he’s got an answer: “536.”
A mysterious fog plunged Europe, the Middle East, and parts of Asia into darkness, day and night—for 18 months… initiating the coldest decade in the past 2300 years. Snow fell that summer in China; crops failed; people starved.
Now, an ultraprecise analysis of ice from a Swiss glacier by a team led by McCormick and glaciologist Paul Mayewski… reported that a cataclysmic volcanic eruption in Iceland spewed ash across the Northern Hemisphere early in 536. Two other massive eruptions followed, in 540 and 547.
The team deciphered this record using a new ultra–high-resolution method, in which a laser carves 120-micron slivers of ice, representing just a few days or weeks of snowfall, along the length of the core… The approach enabled the team to pinpoint storms, volcanic eruptions, and lead pollution down to the month or even less, going back 2000 years
120 microns is roughly the diameter of a single grain of table salt.
-
Apple is introducing automatic narration of select books in their library. I expect this to eventually be an automatic addition to every relevant book on their service although at the moment it appears to require a fair amount of manual review. Notice the “one to two month” lead time.
From Apple.com:
Apple Books digital narration brings together advanced speech synthesis technology with important work by teams of linguists, quality control specialists, and audio engineers to produce high-quality audiobooks from an ebook file.
Our digital voices are created and optimized for specific genres. We’re starting with fiction and romance, and are accepting ebook submissions in these genres.
Once your request is submitted, it takes one to two months to process the book and conduct quality checks. If the digitally narrated audiobook meets our quality and content standards, your audiobook will be ready to publish on the store.
The voice samples at the link above are really impressive. I hope Apple brings these speech synthesis improvements to other parts of their ecosystem. Safari’s built-in text-to-speech feature is shockingly bad in comparison.
-
Dina Bass, reporting for Bloomberg:
Microsoft Corp. is preparing to add OpenAI’s ChatGPT chatbot to its Bing search engine in a bid to lure users from rival Google, according to a person familiar with the plans.
Microsoft is betting that the more conversational and contextual replies to users’ queries will win over search users by supplying better-quality answers beyond links
The Redmond, Washington-based company may roll out the additional feature in the next several months, but it is still weighing both the chatbot’s accuracy and how quickly it can be included in the search engine
Whether or not this succeeds will be determined by the UI decisions Microsoft makes here. I think the best idea, particularly when introducing this as a new interface element, is to frame the AI as an extension of the existing “instant answers” box. Allow the user to ask the AI clarifying questions in the context of their search. Leave the standard search results as they are. Don’t touch anything else. Below is a quick mockup of the UI I am imagining.
Although I am not completely convinced that this will be an overall improvement for web search as a tool I am excited to see how other players respond — especially Google. We may finally start seeing some innovation and experimentation again.
-
Take a moment to consider the following questions before you click:
If you were tasked with designing a building in one of the coldest places in the world what are the factors you should consider? Ice buildup, insulation, frozen pipes… there are a lot! Even if you limit yourself to just the doors. Which direction should they open? How about the door handles? You better make sure nothing freezes shut!
The anonymous writer behind the brr.fyi blog shares their observations from Antartica:
One of the most underrated and fascinating parts of McMurdo is its patchwork evolution over the decades. This is not a master-planned community. Rather, it is a series of organic responses to evolving operational needs.
Nothing more clearly illustrates this than the doors to the buildings. I thought I’d share a collection of my favorite doors, to give a sense of what it’s like on a day-to-day basis doing the most basic task around town: entering and exiting buildings.
-
Petals is an open source project that allows you to run large language models on standard consumer hardware using distributed computing “BitTorrent-style”. From the GitHub repository:
Petals runs large language models like BLOOM-176B collaboratively — you load a small part of the model, then team up with people serving the other parts to run inference or fine-tuning.
In the past I have written about how locally run, open source, large language models will open the door to exciting new projects. This seems like an interesting alternative while we wait for optimizations that would make running these models fully on-device less resource intensive.
-
Pepys' diary is a website, newsletter, and RSS feed that publishes, in real time, diary entries from 17th century civil servant Samuel Pepys' diary. The diary contains first-hand accounts of the Restoration, the Great Plague, and the Great Fire of London as they occur.
Here is a taste of what to expect. With the Fire of London raging Pepys must think fast to save his parmesan cheese. From the September 4th, 1666 entry:
…the fire coming on in that narrow streete, on both sides, with infinite fury. Sir W. Batten not knowing how to remove his wine, did dig a pit in the garden, and laid it in there; …And in the evening Sir W. Pen and I did dig another, and put our wine in it; and I my Parmazan cheese, as well as my wine and some other things.
The current reading just began on January 1st and will conclude in a decade with the final entry published on May 31st, 2033.
-
Happy new year!
§ The Pocket Operator has become my newest obsession. I had forgotten how much I enjoy experimenting with little musical toys. Begin the countdown to when I finally give up and buy an OP-1.
§ I got on a serious weird movie kick this week after watching Triangle of Sadness after Alex Cox and Merlin Mann mentioned it on a recent episode of their Do By Friday podcast.
Triangle of Sadness was alright but I thought The Square, also by Ruben Östlund, was amazing. Although I will admit that some of my enjoyment could be a consequence of going to art school and seeing a lot of hilariously embarrassing aspects of myself in many of the characters.
After watching the two Östlund movies I inevitably had to see The Lobster, a movie I had been avoiding since seeing The Killing of a Sacred Deer a while back and not enjoying it much at all. I ended up loving The Lobster! It might be that Lanthimos writes such amazingly strange, surreal, uncomfortable dialog that I find it all too disturbing in a horror movie but hilarious in a comedy.
§ Re Triangle of Sadness: this song has been stuck in my heard since seeing the movie.
§ There was an unbelievable pink sunset on Wednesday evening that I was actually able to capture a nice photo of. This is usually the type of situation that I find Apple’s computational photography engine “corrects” for, making it really a difficult subject to photograph.
§ Links
- Similar website finder
- Here is an example: sites similar to kottke.org includes dashes.com, waxy.org, scripting.com, and daringfireball.net — pretty good!
- Atlas of Blobs by Zach Lieberman
- Caramel 3.0 — A new beginning
- Awesome ChatGPT
§ Recipes
The unintentional theme this week was cabbage which is definitely a new favorite vegetable — kimchi, sauerkraut, what’s not to like?
- Kapusniak, Polish kielbasa and cabbage soup
- This was one of the best meals I have made in a really long time. Highly recommended. The only thing I will do differently next time is add a liiiitle more chicken broth to thin it out slightly.
- Also, here is a video of Kenji making this recipe
- JANUARY 8 UPDATE: I just realized that when I first made this recipe and wrote the above I had accidentally used half of the specified amount of broth — 4 cups instead of 8 cups — which explains a lot! So now I would suggest either using somewhere around 6 cups of broth or letting the whole soup boil down and condense for a while. Still a great recipe.
- Cabbage Rolls
- This turned out better than I expected but also was more of a pain than it was worth.
- Kimchi soup
- I like spicy foods but this was too spicy for me. It could be the chili flakes I used though. Next time I will either use less chili flakes or a different brand.
- Thai-style beef with basil and chiles
- Not too special but pretty good! I made this one to have with the kimchi soup and it was a good sidekick. Caroline really liked it though.
- Similar website finder
-
John Naughton writes:
2023 looks like being more like 1993 than any other year in recent history. In Spring of that year Marc Andreessen and Eric Bina released Mosaic, the first modern Web browser and suddenly the non-technical world understood what this strange ‘Internet’ thing was for.
We’ve now reached a similar inflection point with something called ‘AI’
The first killer-app of Generative AI has just arrived in the form of ChatGPT… It’s become wildly popular almost overnight — going from zero to a million users in five days. Why? Because everyone can intuitively get that it can do something that they feel is useful but personally find difficult to do themselves. Which means that — finally — they understand what this ‘AI’ thing is for.
-
§ It was super warm outside today. Well, okay, it was 57 °F but for December in Ohio that is about as warm as you could reasonably ask for. Anyway, I very consciously decided that I would not squander the opportunity and spent the afternoon with Caroline collecting willow branches at the park. The goal is to try basket weaving with them soon.
§ Maybe you are like me and you inexplicably decided to keep coturnix quails even though you don’t eat eggs often and now you are constantly inundated with way too many eggs. Well, I have good news for you. Below you will find my inaugural list of Recipes That Use a Lot of Eggs — enjoy!
- French toast 300 grams / ~ 26 quail eggs
- Ice cream 200 grams / ~ 17 quail eggs
- Quiche 300 grams / ~ 26 quail eggs
§ As a part of my ongoing pursuit to make the best chai tea ever, I learned that ginger contains a enzyme called zingibain that makes milk curdle but only if it is added to cold milk. If you wait until your milk to at least 70 °C you’ll be fine! Unless, of course, your goal is to make ginger milk curd.
-
A common belief is that the most valuable proprietary information powering many AI products is carefully engineered prompts and along with parameters for fine tuning. It has become increasingly clear that prompts can be easily reverse engineered, making them entirely accessible to anyone that is interested.
Swyx describes the techniques he used to uncover the source prompts behind each new Notion AI feature and then goes on to argue that treating prompt engineering as a trade secret is the wrong approach. Instead the most important differentiator for AI products is UX:
If you followed this exercise through, you’ve learned everything there is to know about reverse prompt engineering, and should now have a complete set of all source prompts for every Notion AI feature. Any junior dev can take it from here to create a full clone of the Notion AI API, pinging OpenAI GPT3 endpoint with these source prompts and getting similar results as Notion AI does.
Ok, now what? Maybe you learned a little about how Notion makes prompts. But none of this was rocket science… prompts are not moats… There have been some comparisons of prompts to assembly code or SQL, but let me advance another analogy: Prompts are like clientside JavaScript. They are shipped as part of the product, but can be reverse engineered easily
In the past 2 years since GPT3 was launched, a horde of startups and indie hackers have shipped GPT3 wrappers in CLIs, Chrome extensions, and dedicated writing apps; none have felt as natural or intuitive as Notion AI. The long tail of UX fine details matter just as much as the AI model itself…. and that is the subject of good old product design and software engineering. Nothing more, nothing less.
-
Oblomovka on Stable Diffusion:
I understand that people worry that large models built on publicly-available data are basically corporations reselling the Web back to us, but out of all the examples to draw upon to make that point, Stable Diffusion isn’t the best. It’s one of the first examples of a model whose weights are open, and free to reproduce, modify and share
Most importantly, the tool itself is just data; SD 1.0 was about 4.2GiB of floating-point numbers… The ability to learn, condense knowledge, come to new conclusions, and empower people with that new knowledge, is what we do with the shared commonwealth of our creations every day.
Again, I understand if people are worried that, say, Google is going to build tools that only they use to extract money from our shared heritage… [Artists] should be empowered to create amazing works from new tools, just as they did with the camera, the television, the sampler and the VHS recorder, the printer, the photocopier, Photoshop, and the Internet. A 4.2GiB file isn’t a heist of every single artwork on the Internet, and those who think it is are the ones undervaluing their own contributions and creativity. It’s an amazing summary of what we know about art, and everyone should be able to use it to learn, grow, and create.
-
There have been a number of new AI search engine demos released recently. As I have written about before this is an idea I am especially excited about because it can combine the incredible synthesis abilities of LLMs with verifiable citations from the web. This should lead to more accurate mental models of AI and its output — a research assistant, not an all-knowing oracle. This would already be an improvement over Google’s Knowledge Graph which frequently surfaces unsourced answers.
This is a demo inspired by OpenAI WebGPT, not a commercial product. Perplexity Ask is powered by large language models (OpenAI API) and search engines. Accuracy is limited by search results and AI capabilities.
YouChat is a ChatGPT-like AI search assistant that you can talk to right in your search results. It stays up-to-date with the news and cites its sources so that you can feel confident in its answers.
Vik Paruchuri‘s open source Researcher project:
By feeding web context into a large language model, you can improve accuracy and verify the information… Researcher gives you cited sources and more specific information by relying on context from Google.
It is getting increasingly perplexing that we haven’t yet seen any similar search demos from Google or Microsoft.
-
Ben Thompson, from his recent interview with Daniel Gross Nat Friedman:
And text is inherently a good match with deterministic thinking, because you can lay down explicitly what you mean. Yes, you don’t have the person’s internal monologue and thoughts, but they can articulate a whole lot of what is important and get that down on the page and walk you through their logic chain. And text lets you do that to a much greater sense than images. Images, a lot of it is your interpretation of the image. It’s you perceiving what it is. And interestingly, from a biological perspective, vision itself is probabilistic, right? That’s why you get those optical illusions, because your brain is filling in all the different pieces that go into it.
And this makes me wonder, maybe the real difference between deterministic computing and probabilistic computing is in fact the difference between text-based computing or text-based thinking and visual-based thinking. And this visual stuff is in fact not just a toy, it’s not just out there first because it’s easier to do. It actually might be the future of all of this.
-
Qwerty is going to be so sad when I take the tree down.
-
Remember how last week I mentioned that I successfully got through my final week of classes without getting sick? Well, almost immediately after hitting publish I started coming down with a cold. At least it wasn’t especially bad and I was back to feeling nearly 100% by Wednesday.
Anyway, this was my first week off this year and, despite the cold, I ended up accomplishing some of the household tasks I have been putting off for far too long. Most notably, painting a wall in my kitchen and deep cleaning the basement floor where it had been stained by cyanotype chemistry.
Links
- When Technology Changes Art
- Stable Diffusion architecture renders from dollhouse photos
- Related: Depth-to-image in Stable Diffusion
- Analog Diffusion - Stable Diffusion fine tuned on film photographs
- VectorFusion - text to SVG
- picoSYNTH - an interactive synthesizer running in the Pico-8 fantasy console
Recipes
- Tajín grilled chicken
- This was good but would have been better had I used an actual grill instead of the stovetop and broiler. Note to self to try it again next summer.
- Chile crisp fettuccine alfredo
- A better combination than I was expecting. I added lemon juice which was a good choice. Next time I make this I will probably try adding ground pork too.
- Thai carrot and sweet potato soup
- Eh, this was alright but definitely missing something although I am not sure I can pinpoint what exactly
-
Scott Aaronson, in a lecture at the University of Texas at Austin, describes a project he has been working on at OpenAI to watermark GPT output:
My main project so far has been a tool for statistically watermarking the outputs of a text model like GPT. Basically, whenever GPT generates some long text, we want there to be an otherwise unnoticeable secret signal in its choices of words, which you can use to prove later that, yes, this came from GPT.
At its core, GPT is constantly generating a probability distribution over the next token to generate… the OpenAI server then actually samples a token according to that distribution—or some modified version of the distribution, depending on a parameter called “temperature.” As long as the temperature is nonzero, though, there will usually be some randomness in the choice of the next token
So then to watermark, instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI.
In my first two posts on the future of AI in education I highlighted proposals that frame generative AI use as an inevitability and, therefore, a tool to be embraced instead of a threat that warrants the development of technical detection mechanisms. While the risks of plagiarism and misinformation are undoubtedly real, we should push for a greater focus on strengthening critical analysis and fact-checking skills instead of starting an impossible to win arms race between detection and evasion.
The most exciting path forward is one where we frame Large Language Models as “a calculator for text”. Just as the invention of pocket calculators was a giant disruption that forced us to re-evaluate our approach to mathematics education, language models will continue to force us to re-evaluate our approach to research and writing. Done correctly this will open the door for us to learn more quickly, use our time more effectively, and progress further than we possibly could before.
subscribe via RSS