Personal Digital Assistant

If you are ok with a total free Android look at this: http://92.19.232.58:59999/html/shop/phones/index.php they sell smartphones with Replicant preinstalled. (they will donate atleast 20% of the profist. (They told me that in an e-mail.)

I never used a PDA so I can't answer your actual question. I guess a (second hand) PDA might be a decent compromise between a phone and a "smartphone". Looks like there is some palm related software in the repos.apt-cache search palm|sort|grep -i palm

I once considered getting a NanoNote for the job. I gave up that line of thought mostly because I couldn't understand the networking on the device. (no stylus there) https://en.wikipedia.org/wiki/NanoNote

You could try the OpenPandora, or its successor, the Pyra.

I have an OpenPandora. Using it in freedom is a bit difficult, and it's a very weak device when you do (no graphical hardware acceleration, and no wireless of any kind). The way I set it up was to install a 3rd-party OS called "Pandian" (a slightly modified Debian) and purge a package which added the proprietary firmware needed by the wireless controller. To let it connect to the Internet, I use the small wireless adapter sold by Think Penguin. (Funny enough, the OpenPandora's built-in wireless, in addition to not respecting your freedom, doesn't work well at all, so the mini-adapter actually performs much better.)

But once you do manage to install and clean up Pandian, you will have a small (pocketable) computer with a mostly decent keyboard and a touchscreen that you can use either with a finger or with a stylus, running only libre software.

The Pyra is supposed to use Debian by default, so using that in freedom will probably be easier. Asking on the Pyra/OpenPandora forums,[1] I was told that the Pyra would have basically the same amount of respect for the users' freedom as the OpenPandora, with the caveat that full sound quality will require a proprietary firmware blob. So, no hardware acceleration, no wireless, and poor sound quality will be the practical setbacks of making it run entirely libre software.

[1] https://boards.openpandora.org/topic/18273-how-close-does-the-pyras-design-come-to-respecting-your-freedom/

The Pandora uses an older Texas Instruments processor (not Sitara) and I think it uses the u-boot bootloader which is released under the GPL.
No non-free software is required to operate, if you want to use WIFI you can use an external WIFI adapter from ThinkPenguin or Tehnoetic.
Next step would be to port a free distro that one can install in NAND flash. This is quite different than installing to a hard drive.

I thought about libreCMC or ProteanOS, which can be made to fit in less the 512 MB.

i use my nano note
I also have zipit z2 that i use for usb host jobs.
and i also have a gta04 im pending to learn (with help for the chap i
bought it from) about howto compile kerenls for it.

firstly nano notes are very rare now.
same for gta04 but the team is working on its exciting successor
https://neo900.org

but zipit z2’s are easy to get hold of. in fact i bought some (reduced,
as one off deal) from hinv in the usa from the #zipit freenode irc. i
have various bits (like a 2gb minisd card, breakout connectors,etc) and
spare zipits a to make a zipit goodie pack that i can sell for around
£25 GBP + postage. I’d same its great deal when a zipit off evilbay is
£20(inc post). i make no profit. anyway if your interested ill link to
photos and spreadsheet with the list of what you get and the costs. so
you know why the price is what it is.
https://mozzwald.com/zipitz2 for zipit info.

oww pre-libreated replicant phones!!! didnt know about that, ill have a look

I wish the Pyra would take off. I hate smartphones and I would rather have a class of mobile computing that's dead in the consumer market, a UMPC, but I don't think they can enough people willing to spend $1,500 for a dual core machine just because it's a fully open modular platform with a physical keyboard that's the size of a gameboy. It's $1,500 for the prototypes because they can't find enough people willing to spend $1,000 on it because it's such a niche product. (and the designer isn't planning on making much money on it, only a 5% margin.

I think it's niche not because people don't want it, but because people want a powerful phone and don't want to pay for it so they have a contract subsidize it. Also, the market doesn't know there's a viable interest for slider phones, they haven't been selling well because the ones left had crap specs or have the narrow keyboard. But $1,500 sounds like a lot for a Pyra, (and it is) but the way it's designed, it would last me 5 years and $1,500/60 months==$25 per month. I think the creator needs to do something like a patreon and use each individual's contributions as their personal credit towards a Pyra.

I actually just got GNU/Linux working on my Palm T|X a few minutes ago. It seems to be working just fine so far.
There are a ton of dead links on how to do so, and important information is scattered between a ton of different resources. I'll write a tutorial and host a mirror of these fairly hard-to-find files as soon as I get wifi working.