RetroBSD

2.11BSD operating system for microcontrollers
It is currently Wed Nov 14, 2018 4:19 am

All times are UTC




Post new topic Reply to topic  [ 6 posts ] 
Author Message
PostPosted: Mon Apr 21, 2014 9:19 am 
Committer
User avatar

Joined: Thu Oct 11, 2012 8:45 am
Posts: 1801
Location: Room 217, Floor 8, Arm 8, Wheel S7, Mars Base Alpha 3
We really really need to make more efficient use of our limited resources. Most wasted of these currently is flash memory.

I'd like to thrash out any ideas you may have about how we can best use the spare flash memory we have to make programs smaller or allow bigger programs to be run.

I'll start the ball rolling with an idea that's been rattling around my big empty head for a while now...

At the moment all our programs are 100% statically linked. Until we have full elf support that's never going to happen. Much of a programs size comes from library code, such as the code for printf() etc.

If we could load the code for common large library functions into flash, maintain an index of the entry points for said functions, and replace the linked library functions themselves with small stubs that look up the entry point in flash and call that code, then programs that use those functions would kind of have a shared library scenario reducing code size.

The flash code would have to be installed as part of the kernel installation of course, though not necessarily compiled as part of the kernel compilation (though it could be?)

Theoretically it would be completely transparent to any program compiled against the new stub-filled libraries.

There would be some caveats and gotchas though. The flash-based functions would be restricted to a certain extent, as they would be compiled separately to the program that's calling them. Therefore each function would have to be 100% self contained. That means that things like global and local-static variables would be out - everything would have to be be completely re-entrant. Things like strtok() would have to stay in the library, but that doesn't mean to say it couldn't call strtok_r() stored in flash...

Thoughs? Something glaringly obvious I missed? Better ideas?

_________________
Why not visit my shop? http://majenko.co.uk/catalog
Universal IDE: http://uecide.org
"I was trying to find out if it was possible to only eat one Jaffa Cake. I had to abandon the experiment because I ran out of Jaffa Cakes".


Top
 Profile  
 
PostPosted: Mon Apr 21, 2014 10:49 am 
Contributor

Joined: Mon Apr 29, 2013 1:56 am
Posts: 196
majenko wrote:
Thoughs? Something glaringly obvious I missed? Better ideas?


Is that code going to be directly accessible and executable? If it needs to be copied/moved prior to execution, it needs to be position-independent.


Top
 Profile  
 
PostPosted: Mon Apr 21, 2014 11:02 am 
Committer
User avatar

Joined: Thu Oct 11, 2012 8:45 am
Posts: 1801
Location: Room 217, Floor 8, Arm 8, Wheel S7, Mars Base Alpha 3
Yes, it would be directly executed in place in flash. It wouldn't be moved anywhere - it would be static. Installed into flash when you burn the kernel and never change - position independence wouldn't be required for that as it would be compiled to sit at a specific location in the flash and execute at that location. I guess mechanisms could be put in place to update / upgrade that code from userland, but that would come later.

I guess another requirement would be of the flash code being completely atomic - 100% self contained from a code perspective - the only functions ever called from in-flash code would have to also be in-flash code. It wouldn't know where to find the functions in libraries linked into your program. So it could only be incredibly low-level routines that would be put in to flash.

_________________
Why not visit my shop? http://majenko.co.uk/catalog
Universal IDE: http://uecide.org
"I was trying to find out if it was possible to only eat one Jaffa Cake. I had to abandon the experiment because I ran out of Jaffa Cakes".


Top
 Profile  
 
PostPosted: Mon Apr 21, 2014 5:14 pm 
Committer

Joined: Thu Nov 08, 2012 3:55 am
Posts: 177
The uflash branch in svn has examples of compiling and running user programs out of flash, as well as the ability to load the specially compiled programs into flash (although it only supports a single program at any one time). Although it does allow some programs to be run that otherwise would not fit, it seems that most programs do not fit due to a lack of data space, not program space, so the approach is limited.

Putting shared library components in the flash instead of entire programs would be more generally useful, but the limitations concerning static variables seemed like a significant limitation, so I did not persue that option.


Top
 Profile  
 
PostPosted: Mon Apr 21, 2014 5:56 pm 
Contributor
User avatar

Joined: Thu Nov 08, 2012 7:04 am
Posts: 2401
Location: Rapa Nui
Quote:
it seems that most programs do not fit due to a lack of data space, not program space, so the approach is limited.

This is what I am trying to elaborate just now. Imagine you have "fast" ramdisk with fs on it - would it be possible to virtualize somehow the data space there? For example I need to create a 512x512 large int array and process it. Currently it works in memory but max 100x100.
PS: there are compilers which support so called "addressmod" - you have to define a read/write function to any external media, and then you type the variable as ie:
Code:
Ramdisk myarray[512,512];

_________________
Pukao Hats Cleaning Services Ltd.


Top
 Profile  
 
PostPosted: Mon Apr 21, 2014 6:26 pm 
Committer
User avatar

Joined: Thu Oct 11, 2012 8:45 am
Posts: 1801
Location: Room 217, Floor 8, Arm 8, Wheel S7, Mars Base Alpha 3
Quote:
it seems that most programs do not fit due to a lack of data space, not program space, so the approach is limited.

But... program space and data space share the same small chunk of RAM. If you can reduce the program space requirements then you have more RAM available for data space.
Quote:
This is what I am trying to elaborate just now. Imagine you have "fast" ramdisk with fs on it - would it be possible to virtualize somehow the data space there? For example I need to create a 512x512 large int array and process it. Currently it works in memory but max 100x100.
PS: there are compilers which support so called "addressmod" - you have to define a read/write function to any external media, and then you type the variable as ie:
Code:
Ramdisk myarray[512,512];

C++ could do that easily enough, but we have no C++ libs, only C. libvmf is the closest where you define a page of RAM and swap it in and out of your own swap file on disk (could be on an fs on a RAM disk).
Code:
DESCRIPTION
       This library provides a standard set of routines for managing large
       virtual memory spaces.  It supports creation of multiple concurrent
       virtual  spaces,  mapping  of virtual pages  into  real  memory,  a
       lock/unlock mechanism,  and a capability to clear specified virtual
       pages.

... and is already in RetroBSD.

_________________
Why not visit my shop? http://majenko.co.uk/catalog
Universal IDE: http://uecide.org
"I was trying to find out if it was possible to only eat one Jaffa Cake. I had to abandon the experiment because I ran out of Jaffa Cakes".


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
cron




Powered by phpBB® Forum Software © phpBB Group

BSD Daemon used with permission