Hi!

So, my project did result in... something. Nothing too impressive, though. My implementation of search ended up being about the same speed as the current implementation because of the pickle module. I finished my project at the very last minute before it was due, so I didn't have time to find and implement an alternative pickling/serialization module. There must be a faster python pickler out there, though, any recommendations? After I turned in my project I had final exams and then winter break, and I basically didn't want to look at my code at all during that time. Now that you've brought it up, though, I wouldn't mind working on it, perhaps polishing it (okay, it needs more than polishing) so that it might actually be a nice addition to portage? I'm not doing any coding for any of my classes this semester (except for some assembler) so I definitely wouldn't mind working on this on the side.





The reason why it will need (significantly) more work is because I basically had no idea what I was getting into with the regex search. I implemented $ and *, if I remember correctly, and for anything else the search just defaults to the current portage search. I don't know whether implementing regex search with the suffix tree that I used to implement the search would make sense... I'll have to think about it some more. In fact, I have nothing else to do this rainy afternoon :]





If I can find an unpickler that can unpickle at a reasonable speed, my search implementation would be significantly faster than the one currently in use. I'd show you my code, but I have to admit I'm intimidated by Alec's recent picking apart of Doug's code! For example, I don't even know how to use docstrings... The code probably could be cleaned up a lot in general since I was eventually just trying to get it to work before it was due.



Thanks for asking, let me know what you think! (Also, sorry, René, for sending this to you twice.)

Emma
On Thu, Feb 12, 2009 at 2:16 PM, René 'Necoro' Neumann <lists@necoro.eu> wrote:




-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA1



Hey,



has your project resulted in anything?



Just curios about perhaps nice portage additions



Regards,

Necoro



Emma Strubell schrieb:

> Hi everyone. My name is Emma, and I am completely new to this list. I've

> been using Gentoo since 2004, including Portage of course, and before I say

> anything else I'd like to say thanks to everyone for such a kickass package

> management system!!

>

> Anyway, for my final project in my Data Structures & Algorithms class this

> semester, I would like to modify the search functionality in emerge.

> Something I've always noticed about 'emerge -s' or '-S' is that, in general,

> it takes a very long time to perform the searches. (Although, lately it does

> seem to be running faster, specifically on my laptop as opposed to my

> desktop. Strangely, though, it seems that when I do a simple 'emerge -av

> whatever' on my laptop it takes a very long time for emerge to find the

> package and/or determine the dependecies - *whatever it's doing behind that

> spinner. I can definitely go into more detail about this if anyone's

> interested. It's really been puzzling me!) So, as my final project I've

> proposed to improve the time it takes to perform a search using emerge. My

> professor suggested that I look into implementing indexing.

>

> However, I've started looking at the code, and I must admit I'm pretty

> overwhelmed! I don't know where to start. I was wondering if anyone on here

> could give me a quick overview of how the search function currently works,

> an idea as to what could be modified or implemented in order to improve the

> running time of this code, or any tip really as to where I should start or

> what I should start looking at. I'd really appreciate any help or advice!!

>

> Thanks a lot, and keep on making my Debian-using professor jealous :]

> Emma

>



-----BEGIN PGP SIGNATURE-----

Version: GnuPG v2.0.9 (GNU/Linux)

Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org



iEYEARECAAYFAkmUdZIACgkQ4UOg/zhYFuDRQQCfeVXb6uy+wBSKll4MHq54MiyX

VawAn0TWrTBVKuxAPFWpQMvvO3yED5Fs

=dBni

-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Emma Strubell wrote:
> There must be a faster python pickler out there, though, any
> recommendations?

Right off the bat, you might try cPickle. It should work identically,
just faster. Also you can try importing psyco, if it's present it will
try semi-compiling some bits and pieces and *might* offer some speed-ups
(as in, it won't always, for small projects it might actually slow it down).

Mike 5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkmUjy8ACgkQu7rWomwgFXrW6wCfS9zTTgqbhi DyaU1opDJO3BM2
VO4AoIaPQ+t27OnTGh7tBEH/mqYntO/v
=NzDj
-----END PGP SIGNATURE-----

Oh, I meant to say, I was indeed using cPickle. That was my first thought as well, that for some reason pickle was being loaded instead of cPickle, but no.

On Thu, Feb 12, 2009 at 4:05 PM, Mike Auty <ikelos@gentoo.org> wrote:

-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA1



Emma Strubell wrote:

> There must be a faster python pickler out there, though, any

> recommendations?



Right off the bat, you might try cPickle. *It should work identically,

just faster. *Also you can try importing psyco, if it's present it will

try semi-compiling some bits and pieces and *might* offer some speed-ups

(as in, it won't always, for small projects it might actually slow it down).



Mike *5

-----BEGIN PGP SIGNATURE-----

Version: GnuPG v2.0.9 (GNU/Linux)



iEYEARECAAYFAkmUjy8ACgkQu7rWomwgFXrW6wCfS9zTTgqbhi DyaU1opDJO3BM2

VO4AoIaPQ+t27OnTGh7tBEH/mqYntO/v

=NzDj

-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Emma Strubell wrote:
> Hi!
>
> If I can find an unpickler that can unpickle at a reasonable speed, my
> search implementation would be significantly faster than the one currently
> in use. I'd show you my code, but I have to admit I'm intimidated by Alec's
> recent picking apart of Doug's code! For example, I don't even know how to
> use docstrings... The code probably could be cleaned up a lot in general
> since I was eventually just trying to get it to work before it was due.

Please don't be intimidated by it. Code review is one of the best methods to
improve your skills. We all sucked at programming at one time and perhaps we
still suck in anything but our favorite language. But if we are to improve
ourselves we need to spend a lot of time reading and coding and still we will
not always get it right. Furthermore other people learn from our code review,
such as you learnt about docstrings. Here[1] is a quick explanation of them.

Have fun,

Marijn

[1]:http://epydoc.sourceforge.net/docstrings.html

- --
Sarcasm puts the iron in irony, cynicism the steel.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmVd5AACgkQp/VmCx0OL2zIFQCgyJYZve1o6DnBBV/HgRV/gWMc
9NkAoLl0M4NX8l+kgWYY3B1dQQtU0/4k
=p/Pq
-----END PGP SIGNATURE-----