Customizing your page size

You voted for it... You got it!

The most requested DVDAF feature (see the poll): the ability to show more than 50 titles per page. Depending on your membership level you may now select up to 200, 500, or 1,000 listings per page. If all titles in a folder do not fit in a page, you get to see them in multiple pages as usual.

The feature can be accessed by clicking on the icon next where it says "Showing 1 - 50 of 148" in the upper left corner.

Ramblings:

Bandwidth (server and network) like any other limited resource needs to be managed. In the case of the variable page size the issue is that bursts of image requests can overwhelm the picture server. Even at current levels the default distro of Apache chokes and freezes (we tried thttpd but that had other issues). We came back to Apache and changed the MPM ("Multi-Processing Module") plus a few other things and got a stable machine that has been running well for the last year. On a big corporation the solution might have been to throw in a couple more machines, increase system resources, etc. But on a shoestring budget we have to make do with more limited resources.

I considered several ways to address the "variable page size" request.

1. The first idea was to get the page out with a large amount of titles, but only 50-100 pictures. Then Javascript would kick in and load the remaining 100-400 pictures at a reasonable rate. The idea was to give a quick initial response and then to fill in the blanks with a bit more time. We even thought of only requesting images as that particular section of the page came to view. The problem is that if you navigate out of that page and then use the back button Javascript would again go through all that trouble.
   
   
2. The second solution, and the one I liked the most, was to hack some of the Linux networking systems and create two classes of service. For normal people we would give 80% of the bandwidth. For greedy requesters we would give 20%. The system would monitor all TCP-IP traffic and dynamically move people from one class to another as their traffic went up or down. I did a bit of research and the closest thing I found was Bert Hubert's tc. It did not quite fit the bill. We could probably send greedy requests to a different port, tag them in iptables, and pick it in tc. Since I'm not familiar with this stuff it would take me a few weeks to get it working and it would not be browser cache friendly as it would request and cache images from both ports event when they are the same.
   
   [Edit] One important detail is that we cannot deny requests, only slow them down. This means that in order to handle the issue beyond the network layer we would probably hack Apache so that it would respond to requests not in normal order, but in a way that we find suitable -- if not by writing an Apache module, probably by hacking the server itself. It would probably take me a month to do it, but then we might run into maintenance/compatibility issues in the future.
   
   
3. The third solution, and the one I implemented, is very simple. Guests and regular members can see up to 200 pictures per page. Star members ($10 level donation) can see 500. Star members that are even more generous ($30 level donation) can see up to 1,000. The idea is that if a lot of people get to see 500 or 1,000 pictures per page they will be supporting the site with donations and then we would be able to get a new server to handle the load.

What is next:

I got no takers in helping setting up Trac (see previous blog entry), but I think that it is really an important thing. So I'll be splitting my time in setting it up and customizing it, plus the changes identified in 2.1.41.x below.

Release 2.1.42.0 will probably have the second most popular feature request: "Who's got it?"

Thanks!

---

Recent and Upcoming Changes

Release 2.1.41.0 - March 30th
- User configurable page sizes
    + up to 200 for regular members
    + up to 500 for "Supporting" and "Sponsor" members
    + up to 1,000 for "Donor" members and beyond
- Changes in colors for dialog boxes and buttons

Planned for 2.1.41.x
- Moderator's ability to process requests to make a picture the default for a title
- Member's ability to select which of the multiple pictures associated with a title he/she wants to display in his/her collection/listing
- Members ability to see the status of his/her picture submissions